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Overview 


Every year, hundreds of thousands of students enroll at their local community college to earn a 
degree or credential. Their first step upon entering college is to take placement exams in English and 
mathematics to deteimine their readiness to handle college-level courses. Because their scores on 
these tests arc low, over half of entering community college students arc referred to remedial, or 
developmental, courses. Most do not complete the sequence of remedial courses or earn a credential. 

Many community colleges arc turning to learning communities as an intervention to improve the 
outcomes of developmental education students. In learning communities, small cohorts of students 
arc placed together in two or more courses for one semester, usually in the freshman year - . The idea 
behind these communities is that students will be more likely to foim stronger relationships with 
each other and their instructors and engage more deeply in the content of the integrated course work, 
and that this will give them a better chance of passing their courses and staying in college. 

In 2006, the National Center for Postsecondary Research, of which is MDRC is a partner, launched 
a demonstration of one-semester learning community programs at six colleges; five of these 
programs focused on developmental education. This is the final report from the project and includes 
findings from analyses that pool data across these five programs as well as the results for develop- 
mental education students at a sixth program at Kingsborough Community College, operated earlier 
under the Opening Doors demonstration. Across the six programs, almost 7,000 students were 
randomly assigned, about half into 174 learning communities, and tracked for three semesters. Key 
findings suggest that when compared with business as usual, one-semester learning communities in 
developmental education, on average, lead to: 

• A modest (half-credit) estimated impact on credits earned in the targeted subject (English or 
mathematics) but no impact on credits earned outside the targeted subject. 

• A modest (half-credit) estimated impact on total credits earned. 

• No impact on persistence in college. 

The developmental education students in the Kingsborough program, which had some different 
features from the other five programs, including enhanced support services, showed somewhat 
larger results than the other sites in credits earned in the targeted subject. 

An MDRC report on the overall Kingsborough learning communities program, which served both 
developmental and college-ready students, shows a positive impact on degree attainment after six 
years. The graduation effect was driven primarily by students who had placed into college-level 
English, although there is also evidence that the program had a positive impact on long-term 
outcomes for students with the greatest developmental needs in English. Together, these evaluations 
suggest that, while most typical one-semester learning communities for developmental education 
students are not likely to lead to large effects on students’ outcomes, a program with additional 
supports can have longer-term impacts for developmental students. 
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Preface 


Only about a third of the students who enroll in community colleges hoping to earn a degree or 
credential achieve their goal within six years. For those who are not academically prepared for 
college and must take developmental, or remedial, courses in reading, writing, or math, even 
fewer succeed. Many never pass all the required remedial courses and end up leaving school 
without a diploma or certificate. Learning communities are a popular instructional reform 
community colleges are implementing to help developmental education students overcome this 
barrier to finishing college. In developmental education learning communities, small cohorts of 
students are placed together in two or more thematically linked courses, including a develop- 
mental course, usually for one semester. The theory is that the relationships students form with 
each other and their instructors and the connections between their linked courses will enhance 
their engagement with school and their mastery of the subject matter, and that they will be more 
likely to complete their sequence of developmental courses, stay in college, and graduate. 

Recognizing the need for more research on the effectiveness of learning communities, 
the National Center for Postsecondary Research (NCPR), of which MDRC is a partner, 
launched the Learning Communities Demonstration in 2006 to evaluate one-semester learning 
communities at six colleges; five of these learning community programs focused on develop- 
mental education students. Site-specific reports on the effects of the programs at all of the 
colleges are available at www.mdrc.org. This final report on the demonstration analyzes pooled 
data from the five colleges whose programs involved developmental education students and 
also includes developmental students at a sixth program at Kingsborough Community College, 
which was part of MDRC’ s earlier Opening Doors demonstration. These six programs represent 
a range of learning community programs at community colleges. 

After one program semester, the study found that, on average, there was a positive but 
small impact (half a credit) on credits earned in the target subject (English or math), no impact 
on credits earned in other courses, and a half-credit effect on total credits earned. There were no 
effects on semester-to-semester persistence. The programs’ positive impacts during the program 
semester did not grow but were maintained through two postprogram semesters. 

Kingsborough’s program, which linked three courses and offered enhanced counseling 
and tutoring, textbook vouchers, and encouragement to earn more credits in a six-week interses- 
sion, produced an impact on credits earned in the targeted subject (English) that was one credit 
larger than the pooled average. An MDRC report on the six-year follow-up results of Kings- 
borough’s Opening Doors Learning Communities program (which served both developmental 
and college-ready students) found evidence of an effect on graduation rates, although this was 
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driven primarily by students who placed into college-level English. Developmental education 
students with the greatest deficits in English may also have benefited. 

Overall, the analysis in this report reinforces the key findings presented in the previous- 
ly published site-specific reports. On average, one-semester learning communities as typically 
operated for developmental education students should not be expected to produce more than a 
modest impact on credits earned and no effect on persistence. The evidence also suggests that a 
more comprehensive learning community program, such as Kingsborough’s, may lead to 
greater benefits for academically underprepared students than the average program. However, 
programs such as Kingsborough’s are more expensive to run and may be more challenging for 
the typical college to operate at scale. 

To produce dramatic improvements in the outcomes of developmental education stu- 
dents, approaches that are far more comprehensive than a one-semester learning community 
may be required. Progress needs to be made in carrying out and evaluating robust reforms 
aimed at fundamentally transforming the educational experience of academically underprepared 
students. 


Gordon L. Berlin 
President, MDRC 
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Executive Summary 


Every year, hundreds of thousands of Americans — some fresh out of high school, others older 
and working to support themselves or their families, some financially dependent on their 
parents, others parents of their own children — show up at their local community college to 
register for classes. While a few register for a class or two for recreational purposes, the vast 
majority enroll intent on their goal of earning a postsecondary credential so that they may either 
pursue a career or qualify for a better job. Virtually all have a high school credential and believe 
that because they do they can start earning college credits right away. But before they can 
register for classes, they are first required to take a test in both mathematics and English 
(reading and writing). As they file into the testing room, many do not realize that the college has 
identified a “cut score” ahead of time for each test, and if their score falls below it, they will be 
required to take one or more developmental math or English courses before enrolling in college- 
level courses or graduating. These “developmental education students” make up over half of all 
entering community college students. 1 A large proportion of students who are referred to 
developmental education never enroll in or complete their recommended sequence of courses. 2 

Efforts to improve the outcomes of students who are assigned to developmental educa- 
tion in community colleges have intensified in recent years, and rigorous evaluations of these 
are gradually accumulating. Recognizing the need for more research, the U.S. Department of 
Education, through a grant (R305A060010) from the Institute of Education Sciences, estab- 
lished the National Center for Postsecondary Research (NCPR) in 2006. One of NCPR’s 
primary studies, led by MDRC, was the Learning Communities Demonstration. Learning 
communities involve placing students into groups that take two or more “linked” courses 
together, and are a widely used strategy to improve the outcomes of community college students 
in developmental education. 


'Clifford Adelman, Principal Indicators of Student Academic Histories in Postsecondary Education, 
1972-2000 (Washington, D.C.: U.S. Department of Education, Institute of Education Sciences, 2004); Paul 
Attewell, David Lavin, Thurston Domina, and Tania Levey, “New Evidence on College Remediation,” 
(Journal of Higher Education 77, 5: 886-924, 2006). 

2 Thomas Bailey, Dong Wook Jeong and Sung-Woo Cho, Referral, Enrollment, and Completion in Devel- 
opmental Education Sequences in Community Colleges. CCRC Working Paper No. 15 (New York: Communi- 
ty College Research Center, Teachers College, Columbia University, 2009). Most estimates of the completion 
rates of students in developmental education in community colleges are around 20 percent. See; Colleen 
Sommo, Alexander K. Mayer, Timothy Rudd, and Dan Cullinan with Hannah Fresques, Commencement Day: 
Six-Year Effects of a Freshman Learning Community Program at Kingsborough Community College (New 
York: MDRC, 2012); Attewell, Lavin, Domina, and Levey (2006); Davis Jenkins, Shanna Smith Jaggars, 
Matthew Zeidenberg, and Sung-Woo Cho, Strategies for Promoting Gatekeeper Course Success Among 
Students Needing Remediation: Research Report for the Virginia Community College System (New York: 
Community College Research Center, Teachers College, Columbia University, 2009). 
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The primary question addressed in this report is whether learning communities lead to 
better educational outcomes than regular classes, or “business as usual,” for students who are 
placed into developmental English and math at community colleges. The main findings are that 
learning communities produce a small (half-credit) impact on credits earned in the targeted 
subject (English or math), no impact on credits outside that subject, and a small (half-credit) 
effect on total credits earned during up to two semesters after the program ends. They do not 
have an impact on persistence in college. These findings are based on a sample of nearly 7,000 
students who were assigned at random to learning communities or to regular college classes and 
services. The learning communities in the study varied in content and quality but are probably 
typical of how learning communities are usually operated in community colleges. 

What Are Learning Communities? 

Learning communities refer to small cohorts of students who are enrolled together in two or 
more linked courses in a single semester. Unlike instructors of “stand-alone” courses, learning 
community instructors are expected to communicate with each other to align their syllabi, write 
integrated curricula, and prepare common assignments. They also discuss individual students 
who appear to be struggling or whose attendance is erratic in order to come up with a shared 
strategy to get them back on track academically. In addition to the linked classes, learning 
communities often include enhanced support such as tutoring and extra advising. It is thought 
that students in learning communities will form stronger relationships with each other and with 
their instructors, will engage more deeply with the content of the courses when they see a 
context for what they are learning, and will therefore be more likely to pass their courses, persist 
from semester to semester, and graduate with a credential. 

Early Evidence of the Promise of Learning Communities 

Three years before the NCPR and the Learning Communities Demonstration got under way, 
MDRC launched the first major random assignment study of learning communities as part of 
the Opening Doors demonstration. Opening Doors tested several distinct interventions designed 
to promote better student outcomes in community colleges, including a learning communities 
program at Kingsborough Community College in Brooklyn, New York. The program at 
Kingsborough linked a developmental or college-level English course with a college-level 
course such as psychology or sociology, and a freshman orientation course. In addition to 
attending linked classes together as a cohort, students received book vouchers, extra advising, 
and access to a tutor. 

Early results from an evaluation being conducted by MDRC of this program were en- 
couraging. Students who were given the opportunity to enroll in a learning community were 
more likely than the control group to attempt and pass the developmental course sequence in 
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English, and by the end of the fourth semester, students in the program group had earned 2.4 
credits more than students in the control group. 3 It was these findings that provided the impetus 
for both a long-term follow up of the students in the original Opening Doors learning communi- 
ty program at Kingsborough and the multisite Learning Communities Demonstration. 

The Six Learning Communities Programs in This Report 

The Learning Communities Demonstration focused on learning communities designed for 
students in developmental education. Live of the six community colleges that participated in the 
demonstration targeted such students. 4 This report combines data from these five sites as well 
data on a subset of sample members (those who tested into developmental English) from 
Kingsborough’ s Opening Doors program, described above. The resulting six programs studied 
in this report were operated by the following colleges: 

• The Community College of Baltimore County (CCBC) (Baltimore, MD) 

• Hillsborough Community College (Tampa, LL) 

• Houston Community College (Houston, TX) 

• Kingsborough Community College (Brooklyn, NY) 

• Merced College (Merced, CA) 

• Queensborough Community College (Queens, NY) 

Typical learning communities in community colleges for developmental education stu- 
dents link two or three courses, one of which is a course in developmental English or math and 
another that is either a college -level course, another developmental course, or a “student 
success” course. 5 Instructors typically communicate at least once or twice during the semester to 
align and integrate the courses. Support services such as extra tutoring are often added to the 


3 Susan Scrivener, Dan Bloom, Allen LeBlanc, Christina Paxson, Cecilia Elena Rouse, and Colleen Som- 
mo with Jenny Au, Jedediah J. Teres, and Susan Yeh. A Good Start: Two-Year Effects of a Freshmen Learning 
Community Program at Kingsborough Community College (New York: MDRC, 2008). 

4 One college, Kingsborough, created a different type of learning community, building on its Opening 
Doors program but targeting “college -ready” students — that is, students who had completed most if not all of 
any developmental course requirements and had declared a major to prepare them for a specific career. This 
program is not discussed in detail in this report; readers are encouraged to read about this program and its 
results in a separate report. Site-specific findings from the implementation and impacts studies for all six 
programs in the Learning Communities Demonstration have been published in a series of reports that are listed 
at the end of this report. 

5 The curriculum in student success courses focuses on skills needed to succeed in college, with topics 
ranging from time management and study skills to money management and access to financial aid. 
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program. The learning community programs that were implemented and tested at these six 
colleges were purposely selected to represent a range of such typical programs as they exist in 
community colleges. They varied in several dimensions: 

• Targeted subject: Four of the six programs (CCBC, Hillsborough, Kings- 
borough, and Merced) focused on English as the developmental subject, and 
two (Houston and Queensborough) focused on math. 

• Courses linked: Four programs (CCBC, Kingsborough, Merced, and 
Queensborough) included a college-level course in some or all of the links; 
in addition, the CCBC program included a noncredit seminar to support stu- 
dent learning, and the Kingsborough program also linked a one-credit orien- 
tation course. Hillsborough and Houston linked a developmental course with 
a student success course. 

• Emphasis on curricular integration: Two colleges (Kingsborough and 
Merced) emphasized curricular integration across the links from the start of 
the demonstration, while four only did so once the demonstration was under 
way, in part responding to encouragement to do so by MDRC and other col- 
leges. 6 

• Inclusion of additional support services: One college (Kingsborough) 
stood out from the others in the intensity and type of extra academic and ad- 
vising support it offered students in learning communities. Tutors were as- 
signed to each learning community. Kingsborough also helped students in 
the learning communities pay for their books, and if they agreed to enroll in 
the short intersession following the program, they received an additional 
book voucher. 

Overview of Study Design and Key Findings 

The focus of this study is determining the effectiveness of learning communities in improving 
persistence and progress toward a credential compared with regular services offered in devel- 
opmental education. Random assignment was used in all six sites to create a program group of 
students who were given the opportunity to enroll in a learning community and a control group 
of students who could enroll in any course for which they were eligible as long as it was not in a 
learning community. Across the six sites, 6,974 students were randomly assigned, about half of 


6 While an effort was made to include programs that were more advanced (including tight integration be- 
tween the linked classes), as well as more basic (primarily the co-enrollment of students in two or more 
classes), it proved difficult to find programs in the former category. 
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whom had the chance to enroll in one of the 174 learning communities across the six colleges. 
Transcript data were collected documenting enrollment, course-taking, and credit accumulation 
for the semester immediately following random assignment and for two semesters afterward. 
Implementation research was conducted at each site to assess the degree to which the programs 
were implemented with fidelity to the programs as designed by the colleges. 

Findings from the Impact Study Using Pooled Data Across All Sites 

Three semesters after they entered the study, the program group students appeared to 
have slightly better outcomes in terms of credits earned than the control group students, who did 
not have the opportunity to enroll in a learning community. 

• Learning communities had a small, positive effect on progress in the 
subject targeted by the learning community (either English or mathe- 
matics). Learning communities, on average, had a positive impact both on 
students’ attempting and earning credits in the targeted subject, as shown in 
Figure ES.l. The program's estimated half-credit impact in the program se- 
mester was maintained for up to two semesters after the learning community 
was over. The estimated impact on credits earned was in part driven by the 
impact on attempting credits: When students are more likely to enroll in the 
course, they are more likely to complete it. 

• Learning communities did not have an effect on credits earned in other 
courses outside the targeted subject. The program and control group 
earned about the same number of credits in college-level courses, “student 
success” courses, and other developmental courses that were outside the tar- 
geted subject area. 

• Learning communities had a small positive effect on overall academic 
progress (total credits earned). During the program semester, learning 
community students earned an estimated half-credit more than their control 
group counterparts, representing an 8 percent increase in total credit accumu- 
lation. This was primarily a result of students earning half a credit more in 
the targeted subject. Over the following two postprogram semesters, the 
magnitude of the cumulative estimated impact remained approximately the 
same, as shown in Figure ES.2. 

• The programs varied in effectiveness with respect to progress in the tar- 
geted subject area but were fairly homogenous with respect to total 
credits earned. For example, the estimated impact on progress in the target- 
ed subject area at Kingsborough was one credit more than the pooled average 
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The Learning Communities Demonstration 
Figure ES.l 

Cumulative Credits Earned in the Targeted Subject 
by Pooled Sample of Developmental Education Students 

Final Report of the Learning Communities Demonstration 
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SOURCE: MDRC calculations from the Community College of Baltimore County, Hillsborough Community 
College, Houston Community College, Kingsborough Community College, Merced College, and 
Queensborough Community College transcript data. 

NOTES: Rounding may cause slight discrepancies in sums and differences. 

A two-tailed t-test was applied to differences between research groups. Statistical significance levels are 
indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 

The probability of being assigned to the treatment group varies across colleges and within random 
assignment cohorts, and estimates are weighted to account for the different random assignment ratios. Estimates 
are adjusted by campus and cohort. Standard errors are clustered by learning community link. 

impact estimate. However, the programs’ average effects were fairly homog- 
enous across the colleges with respect to total credit accumulation. This sug- 
gests that the pooled results are a reasonable summary of the average effec- 
tiveness of learning communities at these six colleges. 
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The probability of being assigned to the treatment group varies across colleges and within random 
assignment cohorts, and estimates are weighted to account for the different random assignment ratios. Estimates 
are adjusted by campus and cohort. Standard errors are clustered by learning community link. 


Learning communities had no effect on persistence. Students in the pro- 
gram group were no more likely than the control group to enroll in college in 
the first, second, or third semester after they entered the study. 

In addition to examining the overall average effectiveness of learning communities and 
college-level variation in effectiveness, this report considers whether learning communities are 
relatively more or less effective for certain student subgroups. 
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For the main planned subgroups, race by gender and recent high school graduates, there was no 
discernible evidence that learning communities’ effects varied. Based on guidance from a group 
of external reviewers, MDRC conducted exploratory analyses on several additional subgroups, 
including students who are the first in their family to attend college, those who usually speak a 
language other than English at home, single parents, and students whose parents pay for more 
than half of their expenses. For the first three of these subgroups, there was no evidence that 
learning communities led to different impacts. For the fourth subgroup, there was some evi- 
dence that the program may have been more effective for students who were financially 
dependent on their parents (that is, students who reported that their parents paid for more than 
half of their expenses). 

Findings from the Implementation Research and Cost Study 

A review of implementation research conducted in all six sites resulted in the following 
key findings: 

• The programs in all six sites were well implemented according to the in- 
tentions of the colleges, despite considerable variation within each col- 
lege in the extent to which the courses in the links were tightly integrat- 
ed. Even in colleges such as Kingsborough and Merced, which had been 
running learning communities for some time and where integration was em- 
phasized and supported, few learning communities achieved a level of inte- 
gration that would meet the standard of an ideal learning community. 

• The college experiences of students in the program group were on the 
whole distinctly different from those of students in the control group. 

For example, in contrast to the experiences of program group students, stu- 
dents in the control group did not generally attend classes where they knew 
everyone from another class. Learning community instructors were more 
likely than instructors teaching in stand-alone courses to report having beliefs 
and pedagogical practices that are commonly associated with learning com- 
munities. 

• The Kingsborough program and setting differed in ways that may ex- 
plain the somewhat larger impacts at this site. Compared with the other 
sites, Kingsborough offered more enhanced services, particularly advising, to 
students in the learning communities. Also, the three-course link in the learn- 
ing communities was associated with more credits than in the other pro- 
grams, and students in the learning communities were actively encouraged to 
enroll in the short intersession directly after the semester in the learning 
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community. Finally, the students in the Kingsborough program were more 
likely to be full time and financially dependent on their parents. 

• The average cost of a learning community incurred during the semester 
in which the learning community was operated was about $570 per stu- 
dent. The cost per student was higher at Kingsborough than at the other two 
sites for which detailed cost data were collected, owing in part to the provi- 
sion of enhanced support services. 

Conclusions 

Colleges and policymakers now have available a large amount of evidence to consider when 
making critical choices about whether or how to invest in learning communities. This evidence, 
described in this report as well as in a series of reports from both the Learning Communities 
Demonstration and the Opening Doors demonstration, addresses a range of topics, including 
average short-term impacts on academic outcomes, the experience of implementing and scaling 
up learning communities, and the costs of learning communities relative to their effectiveness. 
Importantly, an MDRC report on the Opening Doors program at Kingsborough presents 
evidence that when followed for six years after participating in learning communities, students 
in the Kingsborough sample graduated at a higher rate than the control group . 7 While this was 
most evident for students who had placed into college-level English, there was also evidence of 
long-term impacts for students with the greatest developmental needs in English. 

The overall conclusion from the combined research is that while learning communities 
on average and as typically operated in community colleges for students in developmental 
education are not likely to improve persistence or lead to meaningful impacts on educational 
outcomes, a learning community program with enhancements such as additional advising, book 
vouchers, and the encouragement to stay enrolled continuously may lead to greater benefits . 8 
However, such programs are more expensive to run and may be more challenging for the 
typical college to operate at scale. 

Policymakers and colleges need to have realistic expectations of what a one-semester 
intervention of any kind, including learning communities, can do to change the trajectories of 
large numbers of academically underprepared students. The research on learning communities 
as typically offered in community colleges suggests that, on average, they lead to modest, 
positive effects on credits earned, at least in the short term. However, the road to a credential is 


7 Sommo, Mayer, Rudd, and Cullinan (2012). 
8 Sommo, Mayer, Rudd, and Cullinan (2012). 
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a long one for students who begin in developmental education, and most learning community 
programs alone are not likely to help large numbers of students reach their goal of graduation. 


ES-10 



Chapter 1 

Introduction 


As increasing numbers of Americans seek college degrees, the proportion of students needing 
remediation has increased. Community colleges have become the primary providers of devel- 
opmental education for underprepared college students . 1 Recent years have also seen a growing 
wealth of evidence that the majority of students who are assigned to developmental education 
courses never reach college-level courses in their subject of remedial need and never graduate 
from college with a diploma or certificate . 2 In response to these low rates of completion, 
community colleges have launched a commensurate wealth of programs designed to remedy 
what has come to be known as “the problem of developmental education .” 3 

But until very recently, practitioners, policymakers, and other stakeholders have had 
limited evidence available to them on the effectiveness of developmental education reforms in 
improving students’ educational outcomes . 4 This report adds to the knowledge base, offering 
reliable evidence from multisite experimental studies of learning communities, a popular 
intervention for students in developmental education. Developmental education learning 
communities consist of groups of students who are enrolled together in a developmental class 
and another one or two classes. These linked classes often incorporate shared assignments and 
curricula, collaboration between faculty teaching pairs, and connections to student support 
services. 

This report is the final in a series of reports from the Learning Communities Demon- 
stration. It synthesizes and expands on earlier findings from the demonstration and also includes 
findings from a prior randomized controlled trial of developmental education learning commu- 
nities conducted as part of the Opening Doors demonstration at Kingsborough Community 
College. These studies together provide the most extensive evidence available on the promise 
and limitations of learning communities for improving the academic outcomes of students in 
developmental courses. 

The Policy Context 

Across the United States, community colleges provide millions of students with open access to 
a low-cost college education. Each year, over one -third of the country’s postsecondary enrollees 


'Lewis and Farris (1996); Wirt et al. (2004); Brock (2010). 

“Attewell, Lavin, Domina, and Levey (2006); Roksa et al. (2009); Bailey, Jeong, and Cho (2010). 
3 National Center for Postsecondary Research Developmental Education Conference (2010). 
4 Zachry Rutschow and Schneider (201 1). 
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— approximately 6.7 million students — attend community colleges. 5 However, of first-time 
students who enroll in community college, only about a third earn a degree or certificate within 
six years. 6 While the rates of degree or certificate attainment are low in general, they are even 
lower for students in need of developmental education, who comprise a significant proportion 
of the community college student body. 7 

Whether a student is required to take developmental education courses is typically de- 
termined when she or he first enrolls. Entering students take one or more placement tests to 
determine their basic skills proficiency; if they score above their college’ s cut score (or have 
already demonstrated their skills through high school achievement or standardized tests), they 
may begin in college-level courses; if they fail to demonstrate proficiency, they are referred to 
developmental reading, writing, and/or math. These students form a diverse group, but — 
compared with the full college student body — young, black, Hispanic, and female students 
tend to require more developmental education, as do students who attend part time. 8 Students 
referred to developmental classes are also more likely than others to report that they may need 
to drop a class or withdraw from college because they work full time, care for dependents, or 
lack finances. 9 

Of the students who test into developmental-level reading, writing, or math, fewer than 
half complete the sequence of assigned developmental courses within three years. This number 
is dramatically smaller for students who test at the lowest levels. 10 There are a number of 
reasons for students’ failure to complete these courses, but often it is because they never enroll 
in the recommended courses, rather than because they fail them or withdraw from courses 
midsemester. 11 

Some researchers and practitioners argue that shortening or doing away with the devel- 
opmental education sequence might propel students toward higher rates of degree or certificate 
attainment. 12 Others suggest that encouraging or requiring students to enroll in developmental 
courses early in their college careers will both give them the foundational skills they need and 
put them on the right track toward completing the courses necessary to finish college. 13 Leam- 


5 U.S. Department of Education (201 1). 

6 Radford, Berkner, Wheeless, and Shepherd (2010). 

7 Adelman (2004); Attewell, Lavin, Domina, and Levey. (2006). 
8 Bailey, Jeong, and Cho (2009). 

9 Center for Community College Student Engagement (2011). 
10 Bailey, Jeong, and Cho (2009). 

1 'Bailey, Jeong, and Cho (2009). 

12 Hern (2010); Edgecombe (2011). 

13 Moore and Shulock (2007). 
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ing communities are offered as a strategy to enroll students in developmental education that has 
the potential to improve their longer-term outcomes. 

Recognizing the need for rigorous evidence of the effectiveness of programs and poli- 
cies designed to help students make the transition to college and master the basic skills needed 
to advance to a degree, the Institute of Education Sciences (IES) of the U.S. Department of 
Education established the National Center for Postsecondary Research (NCPR) through a grant 
(R305A060010). NCPR was launched in 2006 as a partnership of the Community College 
Research Center at Columbia University’s Teachers College, MDRC, the Curry School of 
Education at the University of Virginia, and faculty at Harvard University. One of NCPR's 
keystone studies is the Learning Communities Demonstration, focused primarily on determin- 
ing the impacts of learning communities on the educational outcomes of students who have 
been referred to developmental reading, writing, or math. 

Learning Communities as a Lever for Change 

Learning communities are an old strategy intended to improve the educational experience of 
college students, with roots dating to the early 20th century. They have existed in community 
colleges since at least the 1970s. 14 While the exact number of learning communities or colleges 
with learning community programs is not known, the strategy is popular: Over half of 288 
community colleges in a recent survey reported offering learning communities at the develop- 
mental or college level. Only two of these colleges required learning communities for all first- 
time students, 15 and most colleges probably offered only a handful of links. In a related survey 
of community college students, 13 percent reported having participated in a learning community 
at some point in their college experience. 16 

The theory of change for learning communities — at both the developmental and col- 
lege levels — predicts that students in them will become more engaged in what they are 
learning and more connected with each other and with their instructors. The theory predicts that 
as a result of the interdisciplinary connections the instructors emphasize and deeper engage- 
ment, students are more likely to develop higher-order thinking skills, master the course 
material, pass their classes, and persist from semester to semester. 17 The compelling and 
intuitively appealing nature of this theory of learning communities and promising early research 
on their potential effects, coupled with the growing recognition that developmental education 


14 For a discussion of the history of learning communities, see Smith, MacGregor, Matthews, and 
Gabelnick (2004), as referenced in Visher, Schneider, Wathington, and Collado (2010). 
l5 Center for Community College Student Engagement (2012). 
l6 Center for Community College Student Engagement (2012). 

17 See Visher, Wathington, Richburg-Hayes, and Schneider (2008) and Smith, MacGregor, Matthews, and 
Gabelnick (2004) for a review of the literature. 
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poses a formidable barrier to college graduation for many community college students, has led 
learning community proponents to view them as a particularly promising program for students 
in need of developmental education. 18 

The typical learning community model, regardless of the courses in the link, consists of 
four key components, although considerable variation exists in both how much these compo- 
nents are emphasized and how well they are actually implemented in colleges. 19 Table 1.1 lists 
the four components, along with indicators of basic, midrange, and advanced versions of each. 
An advanced program model would include an advanced version of all four components 
described; a basic model would include the most basic versions or only a subset of the compo- 
nents listed. Below are brief descriptions of the four components and how they fit into the 
overall theory of change of learning communities. 20 

Linked Courses and Student Cohorts 

Learning communities consist of groups of students who enroll together into two or 
more courses, often scheduled back-to-back. In basic learning communities, most students are 
enrolled in both courses together, although some mixing with non-leaming-community students 
may occur. In addition, the connection between the two courses may not always be made 
explicit with a common theme or set of goals. In more advanced learning communities, the 
linked courses are chosen deliberately to facilitate teaching that emphasizes the interconnections 
between the two courses, and many learning communities are centered on a common theme that 
will be the focus of the semester. The theory of change posits that students enrolled together in 
linked courses experience an increased sense of support from their fellow students and their 
instructors, stronger engagement with the college community, and a feeling of accountability. 
These results should then increase rates of retention and reenrollment in the next semester. 

Faculty Collaboration 

As part of a learning community, instructors of the linked courses collaborate to plan 
and run their classes. In basic learning communities, although courses are linked in name, 
instructors rarely collaborate on course content or on the progress of their shared students. In 
more advanced learning communities, instructors meet at various points before and during the 
semester. This collaboration between instructors helps develop the pedagogical practices that 
are considered to be central to the theory of change for learning communities and holds students 


18 See, for example. Center for Student Success (2007); and Grubb (2001). 

19 Tinto (1997); Tinto (1998); Engstrom and Tinto (2008); Malnarich (2003); Visher, Schneider, 
Wathington, and Collado (2010). 

20 For a more detailed description of the practices associated with each component, see Visher, Schneider, 
Wathington, and Collado (2010). 
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The Learning Communities Demonstration 

Table 1.1 

Components of the Learning Community Model 
Final Report of the Learning Communities Demonstration 


Component 

Basic 

Midrange 

Advanced 

Linked courses 
and student 
cohorts 

Students in the links 
are a mix of learning 
community students 
and students taking 
the course as a stand- 
alone. 

Most, but not all, of 
the students in the 
linked courses are in 
the learning 
community. 

Cohorts are “pure”: all students 
in all the links are part of the 
learning community. Courses 
are selected to promote 
integration. 

Faculty 

collaboration 

Teacher teams rarely 
communicate about 
curriculum or other 
students. 

Teacher teams 
communicate 
periodically 
throughout the 
semester. 

Teacher teams spend time 
planning before, during, and 
after each semester. Instructors 
have access to professional 
development and ongoing 
support. 

Instructional 

practices 

Courses are taught as 
if they were stand- 
alone. 

Teachers assign at 
least one joint project 
during the semester. 

Syllabi are fully aligned, with 
an overarching theme; 
intentional integration, 
including several joint projects; 
joint grading rubrics; and joint 
attendance policies. Instruction 
includes project-based work 
and group work. 

Student support 

No extra support 
offered to students 
beyond what is 
normally offered. 

Some extra support is 
offered but it is not 
integrated into the 
classroom. 

Extra support is available and 
often integrated into the 
classroom or required for 
students. 


SOURCES: The indicators of advanced learning communities draw on work by Tinto ( 1997, 1998), 
Malnarich (2003), and Smith, MacGregor, Matthews, and Gabelnick (2004). 


accountable for their attendance and behavior in both linked courses. The theory of change 
supporting faculty collaboration suggests that it promotes the deeper learning that occurs in 
learning communities and enhances students’ sense of support in the college community. Some 
colleges offer paid release time or a stipend as well as professional development to support 
instructors as they implement the learning community model. 
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Instructional Practices 

Teaching methods in a learning community are focused on integrated instruction, which 
emphasizes the interconnections between the two linked courses, and active and collaborative 
learning, which includes strategies like group work and hands-on assignments. Instructors try to 
foster integrated learning by connecting course content both with the other course in the link 
and with students’ own lives and the world around them. In a basic learning community, 
instructors might teach each course as if it were a stand-alone section, but in advanced learning 
communities, instructors assign group work and cross-course projects, write joint lesson plans, 
align their syllabi across the two courses, and emphasize relevant connections throughout the 
semester. For example, an advanced learning community could have a common title and theme 
such as “Write About Life, Learn How to Live” (in a link with English and health courses) or 
“A Collection of Mathematical Adventures” (in a link with English and math courses). 
According to the theory of change, this form of contextualized, integrated learning helps 
students connect with the material, engenders deeper learning, and increases students’ academic 
success in the learning community and other courses. 

Student Supports 

Many learning communities programs include enhanced student support services that 
provide students with increased access to tutors, designated counselors, or supplemental 
instruction in the classroom. Basic learning communities provide the same access to services 
that all students receive. Advanced learning communities may offer extra, dedicated support 
services or integrate additional help into the classroom. Some learning communities include a 
student success course as part of the link, thus embedding extra support into the learning 
community itself. The curriculum in student success courses focuses on skills needed to succeed 
in college, with topics ranging from time management and study skills to money management 
and access to financial aid. The theory of change suggests that students who receive enhanced 
support services, especially when they are integrated into the classroom, are less likely to drop 
out of college, more likely to return in subsequent semesters, and more likely to succeed 
academically. 

Although the majority of learning communities in community colleges last only for one 
semester, the theory of change suggests that the sense of engagement and support students feel 
on campus may be powerful enough to increase their likelihood of continued enrollment from 
semester to semester. Furthermore, for students in need of developmental education, increased 
basic reading, writing, or math skills as a result of integrated learning may better prepare them 
for college-level work in a range of subjects. These factors are theorized to lead to increases in 
total credit accumulation and graduation and transfer rates, compared with what would have 
occurred without the program. 
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Research on Learning Communities 

Most research on learning communities is relatively recent and includes important theoretical 
work, some nonexperimental studies, and a few quasi-experimental or rigorous evaluations of 
the effects of learning communities on psychological and academic outcomes. 

Consistent with the theory of change, the research on learning communities provides 
some evidence that participation in a learning community is related to students reporting a range 
of positive social and psychological outcomes. These reported benefits of learning communities 
include more positive feelings about the college experience, increased levels of engagement, 
more meaningful relationships among students, increased interaction around academic activity, 
stronger recognition of connections across disciplines, and improved higher-order thinking 
skills. 21 


Academic outcomes have also been evaluated in nonexperimental and quasi- 
experimental studies. These studies suggest that learning communities may have modest 
positive associations with outcomes such as course completion, grades, and persistence in 
college. 22 For example, students in learning communities at two community colleges were 
found to be more likely to enroll in school the following semester than students who took 
similar courses outside of the learning community program. 23 In subsequent quasi-experimental 
research using longitudinal survey data at 13 community colleges across the country, learning 
communities were estimated to increase the rate of college enrollment for students in 
developmental education by 5 percentage points one year after participation in the learning 
community. 24 While these findings are promising, the designs of these studies leave open the 
question of whether their effects are due to the program itself or to preprogram differences in 
the characteristics of those students who chose to enroll in the program (such as their academic 
ability, motivation levels, or tenacity). 

Randomized Controlled Trials of Developmental Education 
Learning Communities 

The first large-scale randomized controlled trial evaluation of learning communities was 
conducted at Kingsborough Community College as part of the Opening Doors demonstration. 25 
In this evaluation, students who volunteered to be part of the study were randomly assigned to 

2l Engstrom and Tinto (2008); Stefanou and Salisbury-Glennon (2002); Tinto (1997); Tinto and Goodsell 
( 1993); Zhao and Kuh (2004). Only one of these studies — Engstrom and Tinto (2008) — focused exclusively 
on sUidents in developmental education. 

22 Gordon, Young, and Kalianov (2001); Tinto (1997); Tinto (1998); EngsUom and Tinto (2008). 

23 Tinto ( 1997); Tinto (1998). 

24 Engstrom and Tinto (2007); EngsUom and Tinto (2008). 

25 Scrivener et al. (2008). 
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one of two groups: a program group, in which students had the opportunity to enroll in a 
learning community that fit their schedule and course needs, or a control group, in which they 
were allowed to enroll in any course they were eligible for or that was required as long as it was 
not in a learning community. This method of random assignment of students creates a study 
sample comprising two groups that come from the same target population and are similar in 
terms of observable and unobservable characteristics. Measured differences in the educational 
outcomes of students who were assigned to the program and control groups can then be 
attributed with a high degree of confidence to the learning communities intervention, rather than 
to preexisting differences between those who experienced the program and those who did not. 

Promising early findings in the Opening Doors study at Kingsborough (as described 
below) led to the launch of the Learning Communities Demonstration, in which various models 
of learning communities were implemented and tested. The Learning Communities 
Demonstration was originally intended to include at least one or two colleges with programs 
that consistently implemented advanced components across most or all of their learning 
communities and faculty teams. However, while individual examples of advanced learning 
communities were found, it proved difficult to identify colleges that offered programs where 
these were the norm rather than the exception. Learning communities programs with tightly 
integrated courses were particularly hard to find. Even at colleges where curricular integration 
had been emphasized for many years, it was rare to find more than a few very well-integrated 
learning communities. 

Ultimately, five colleges were selected to represent a range of developmental education 
learning community programs that may be typically offered in community colleges . 26 The 
models studied range from the more basic/midrange (where courses are linked and students are 
enrolled together, instructors have at least some contact with each other, and students may 
receive some extra support), to more midrange or advanced (where courses are linked, faculty 
work together to offer joint assignments or projects, and significantly more intensive supports 
are provided than is the case for traditional classes). In the programs studied — as at many 
colleges — there was wide variation among the learning communities offered within each 
college’s program; among many dimensions there was as much variation within each college as 
there was between colleges, particularly in the levels of faculty collaboration and curricular 
integration. 


26 The site recruitment process is detailed in Visher, Schneider, Wathington, and Collado (2010). In addi- 
tion to the developmental education learning communities discussed in this report, Kingsborough Community 
College also ran a program of career-focused learning communities that did not include developmental courses. 
These career-focused learning communities were also evaluated as part of the Learning Communities Demon- 
stration. The results are detailed in Visher and Teres (201 1). 



At the six sites across the two demonstrations, a total of 6,974 students who tested into 
developmental math or developmental English (either reading or writing) were recruited for the 
studies. This report focuses on these students and the 174 developmental education learning 
communities that were created to serve them. 

Students who entered the studies at these colleges were randomly assigned to either a 
group in which they could enroll in a learning community that included one or more 
developmental courses, or to a control group in which they could take their developmental and 
other courses as standard courses offered at the college. 27 Academic outcomes of students in the 
two groups were then compared to learn whether the opportunity to participate in a learning 
community had an impact on students’ rates of attempting or passing courses in the 
developmental sequence, rates of attempting or passing other courses, overall credit 
accumulation, and the likelihood of enrolling in college in subsequent semesters. 

The colleges, programs, and early findings in each of these randomized controlled trial 
studies are detailed in previously released reports and are described briefly below. Table 1.2 
presents information about the institutions and their students during the first year of random 
assignment. Concise descriptions of the program models are presented in Table 1.3, and an 
overview of each program’s previously reported impacts on key outcomes is presented in Table 
1.4. 


• The Community College of Baltimore County (CCBC) comprises three 
main campuses and three additional locations in suburban Maryland, near the 
city of Baltimore. It serves about 20,000 students each fall, with large white 
and black student populations. At CCBC, as at each of the six colleges, the 
majority of students are female. Both the Catonsville and the Essex campuses 
participated in the Learning Communities Demonstration. 

CCBC implemented learning communities that linked developmental 
English (reading or writing) with a college-level course and a weekly one- 
hour Master Learner session designed to support curricular integration and 
student learning. At CCBC, developmental English courses are mandatory 
for all students who test into them, and students in learning communities 
attempted — and passed — these courses at the same rate as students in the 
control group. The program had no meaningful impacts on the number of 
credits attempted, the number of credits earned, or progress through the 


27 For a description of the methodology of the Learning Communities Demonstration, see Visher, Wath- 
ington, Richburg-Hayes, and Schneider (2008). 
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The Learning Communities Demonstration 
Table 1.2 


Selected Characteristics of All Students, by College 
Final Report of the Learning Communities Demonstration 



CCBC 

Hillsborough 

Houston 

Kingsborough 

Merced 

Queensborough 

Institution size 

Undergraduate characteristics 

19,426 

22,621 

43,518 

14,943 

10,890 

13,359 

Gender (%) 

Male 

37.0 

41.7 

40.7 

40.2 

38.9 

43.1 

Female 

63.0 

58.3 

59.3 

59.8 

61.1 

56.9 

Age (%) 

24 and younger 

56.7 

63.8 

55.7 

73.2 

57.2 

75.9 

25-34 

21.7 

21.9 

27.7 

14.0 

19.3 

14.4 

35 and older 

21.6 

14.1 

16.6 

12.7 

23.5 

9.7 

Race/ethnicity (%) 

Hispanic 

2.1 

22.4 

27.6 

14.0 

39.4 

22.0 

White 

55.4 

52.2 

20.0 

40.1 

35.2 

23.8 

Black 

30.9 

18.6 

25.9 

31.1 

5.7 

24.2 

Asian 

4.2 

4.1 

11.1 

10.4 

10.9 

20.4 

Other 

7.4 

2.7 

15.4 

4.4 

8.9 

9.6 

Status (%) 

Full time 

34.3 

32.6 

29.3 

50.1 

39.9 

52.1 

Part time 

65.7 

67.4 

70.7 

49.9 

60.1 

47.9 

Retention rates (%) a 

Full time 

59 

65 

58 

68 

35 

70 

Part time 

42 

47 

46 

49 

26 

56 


SOURCE: MDRC calculations from U.S. Department of Education, National Center for Education Statistics, 
Integrated Postsecondary Education Data System (IPEDS) data. 

NOTES: Data are from fall 2003 for Kingsborough and fall 2007 for all other colleges. Distributions may not 
add to 100 percent because of rounding. 

a According to IPEDS, this is the percentage of first- time degree/certificate-seeking students from the 
previous fall who either reenrolled or successfully completed their program by the current fall. 


developmental English course sequence in the program semester or the 
subsequent semester. The evaluation also did not find impacts on rates of 
reenrollment in college . 28 


28 Weissman et al. (2012). Throughout this report, reenrollment is defined as the percentage of the full 
sample enrolled in subsequent semesters at the end of the add/drop period; this measure is analogous to a 
common definition of persistence. 
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The Learning Communities Demonstration 
Table 1.3 

Overview of Developmental Education Learning Communities, by College 


Final Report of the Learning Communities Demonstration 


College 

Learning Community Program Model 

Semesters of 
Study Intake 

The Community 
College of 
Baltimore County 
(CCBC) 

(Baltimore, MD) 

• Developmental reading or writing linked with a college-level course 
(for example, psychology, sociology, speech) 

• Master Learner Component — a faculty member (sometimes the 
developmental English instructor) sat in on a college-level course and 
conducted a weekly, one-hour, noncredit seminar on learning-to-learn 
in the context of the college-level course 

Spring 2008 - 
Fall 2009 

Hillsborough 
Community 
College 
(Tampa, FL) 

• Developmental reading linked with a student success course 

• Student success course focused on acclimation to college and study 
skills 

Fall 2007 - Fall 
2008 

Houston 
Community 
College 
(Houston, TX) 

• Developmental math linked with a student success course 

• Student success course focused on acclimation to college and study 
skills 

Spring 2008 - 
Fall 2009 

Kingsborough 

Community 

College 

(Brooklyn, NY) 

• Developmental English linked with a college-level course in the 
student's major and a one-credit freshman orientation course 

• Program also included enhanced advising, tutoring, and a textbook 
voucher 

Fall 2003 - 
Spring 2005 

Merced College 
(Merced, CA) 

• Developmental writing linked with developmental reading or math, a 
college-level course, or a student success course 

• Links included cross-content themes and integrated assignments 
developed by the learning community instructor pairs before the start 
of each semester 

Spring 2008 - 
Fall 2009 

Queensborough 
Community 
College 
(Queens, NY) 

• Developmental math linked with developmental or college-level 
English (fall 2007) or with a college-level course (spring 2008 and 
beyond) 

Fall 2007 - 
Spring 2009 


SOURCES: Scrivener et al. (2008); Weiss, Visher, and Wathington (2010); Weissman et al. (2011); Weissman et 
al. (2012). 
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• Hillsborough Community College is an urban community college located 
in Tampa, Florida. Like CCBC, Hillsborough serves approximately 20,000 
students each fall. Three of the college’s five campuses — Dale Mabry, Ybor 
City, and Brandon — participated in the Learning Communities 
Demonstration. Across the college, just over half of the students are white, 
with black and Hispanic students each making up around 20 percent of the 
student population. Thirty-six percent of all students are age 25 or over, and 
almost 70 percent attend college part time. 

Hillsborough offered a learning community model that paired a 
developmental reading course and a student success course. Overall, similar 
to CCBC, the program did not have meaningful impacts on the number of 
credits attempted or earned during the program semester, the likelihood of 
completing a developmental reading course during the program semester, 
total credit accumulation, or college enrollment in the postprogram 
semester. 29 

• Houston Community College, the largest college in the study, is a 
community college system comprised of six colleges located in and around 
Houston, Texas. These colleges, several of which have multiple campuses, 
serve over 40,000 students each fall. Three of the campuses — Central, 
Northline, and Southeast — participated in the Learning Communities 
Demonstration. The student population is almost equal parts Hispanic, black, 
and white, and a majority of students attend part time (7 1 percent, the highest 
proportion across all six colleges). 

Houston offered learning communities for students in the lowest level of 
developmental math and linked this course with the college’ s student success 
course. Learning community students were significantly more likely than 
those in the control group to pass developmental math in the program 
semester and to attempt the second math class in their sequence during the 
next semester. However, there was no significant increase in the percentage 
who passed this second math class in the sequence by the end of a one-year 
follow-up period. The program did not have an impact on cumulative credits 
earned or reenrollment in college. 30 


29 Weiss, Visher, and Wathington (2010). 
30 Weissman et al. (2011). 
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• Kingsborough Community College, located in Brooklyn, New York, is one 
of seven community colleges in the City University of New York (CUNY) 
system. Kingsborough’ s student enrollment each fall is approximately 
15,000, and the student body more or less mirrors the demographics of 
Brooklyn: Forty percent of students are white, 31 percent black, 14 percent 
Hispanic, and 10 percent Asian. The student body is younger than that of 
many other community colleges (nearly three-fourths of Kingsborough 
students are under age 25), and a large proportion of students (about half) 
attend full time. 

Kingsborough’ s Opening Doors learning communities placed freshmen into 
three linked courses: an English course (either writing or combined reading 
and writing, usually at the developmental level), an academic course required 
for the student’s major, and a one-credit student success course. 31 The 
program also provided enhanced counseling and tutoring as well as a 
voucher for textbooks. A student survey found that students in the program 
group felt more integrated and engaged in school than students in the control 
group. The program moved students more quickly through developmental 
English requirements and generated an increase in the number of credits 
attempted and earned in the program semester. At the end of a two-year 
follow-up period, this progress in credits earned was generally sustained, and 
students in the learning communities group were more likely than their 
control group counterparts to be attending college. 32 

• Merced College is a midsized college in Merced, in California’ s agricultural 
Central Valley. Merced has large Hispanic and white student populations and 
serves about 11,000 students each fall, making it the smallest college 
discussed in this report. The college’s retention rates for both full-time and 
part-time students were markedly lower than those at the five other schools, 
with 35 percent of full-time students and 26 percent of part-time students 
reenrolling or completing their program the following fall. 

Merced implemented learning communities that linked developmental 
writing with a variety of other courses at the developmental and college 
levels. Students in learning communities attempted and earned more 


31 A minority of the learning communities studied in the Opening Doors demonstration at Kingsborough 
did not include developmental education courses. Those learning communities are excluded from the analyses 
in this report. 

32 Scrivener et al. (2008). 
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developmental English (reading and writing) credits during the program se- 
semester. In the semester following the program, there were no significant 
differences in the credits that students in the program and control groups at- 
attempted or earned, but cumulatively, students in the program group had 
passed more English courses than their control group counterparts and 
remained ahead in the English sequence as a result of the credit advantage 
gained in the program semester. On average, there was no significant impact 
on cumulative credits earned or on college enrollment following the program 
semester. 33 

• Queensborough Community College, a midsized college in Queens, New 
York, serves over 13,000 students each fall and, like Kingsborough, is part of 
the CUNY system. Queens is home to a very diverse population, and the 
demographic makeup of Queensborough’ s student body reflects this, as it is 
almost equal parts black, white, Hispanic, and Asian. Like Kingsborough, 
Queensborough has a relatively young student body and a relatively high 
proportion of students who attend college full time: Three-quarters of the 
students are under 25 years old, and more than half attend full time. The two 
CUNY colleges also have relatively high retention rates, compared with the 
other colleges discussed in this report. 

Queensborough ran learning communities in which all levels of 
developmental math were paired primarily with college-level courses. As at 
Houston, learning communities led students to take and pass developmental 
math earlier in their college careers; learning community students at 
Queensborough were significantly more likely than control group students to 
pass developmental math in the program semester and to pass the second 
math class in their sequence during the next semester. However, this impact 
generally did not translate into increased cumulative progress in the math 
sequence by the end of the three-semester follow-up period. The program led 
to a moderate increase in credits earned in the program semester, but had no 
impact on the cumulative credits that students earned by the end of the study 
or on enrollment in college at the end of the follow-up period. 34 

Although these results vary somewhat for the different colleges and types of learning 
communities, in general the findings described above and published in earlier reports do not 
point to large academic impacts of learning communities. This prior evidence suggested that, 


33 Weissman et al. (2012). 
34 Weissman et al. (2011). 
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compared with regular services, one-semester learning communities for students in develop- 
mental courses may have an impact on the number of credits students earn in the targeted 
subject area in the program semester, but are typically not sufficient to boost enrollment rates or 
lead to increases in credit accumulation that last beyond one or two semesters after the program. 

Contents of This Report: Additional Follow-Up and Synthesis 
Across the Studies 

Given these results, readers may wonder why more analysis of the data is useful. There are 
several reasons. First, the prior results are based on a relatively short follow-up period that 
varied across the six sites. For example, the Kingsborough results are based on four semesters 
of follow-up (including the semester immediately following random assignment); the results 
from Queensborough are based on three semesters of follow-up; and the results for the remain- 
ing four sites are based primarily on only two semesters of follow-up. This report examines 
outcomes for three semesters of follow-up for all six sites. Second, the outcomes that were 
measured and reported in earlier reports across the six studies varied slightly to align as closely 
as possible with the model and the college’s theory of change. This report uses standard 
measures of outcomes across all sites. Third, pooling data across the six colleges improves the 
precision of estimates of the effects of this type of program, enabling the detection of fairly 
small impacts. This is especially important for discerning differences in impacts across the six 
colleges as well as for subgroups of students. 

This report presents additional data and synthesizes the findings across all six existing 
random assignment studies of learning communities for students in developmental reading, 
writing, or math. As such, it provides a rigorous assessment of the average effects of one- 
semester learning communities on improving students’ educational outcomes across a range of 
community colleges and programs. 

Chapter 2 describes the students at the six colleges who entered the studies. It also pro- 
vides more detail on the study’ s design and on the data sources used in the report. 

In Chapter 3, data are presented for an additional semester of follow-up at four of the 
six colleges, providing three full semesters of follow-up for each college. Data are then pooled 
across the six programs to answer the following questions about the impacts of learning com- 
munities for students in developmental education that cannot be addressed using site-specific 
impact estimates alone: On average across all six programs, do students who are offered the 
opportunity to participate in learning communities make greater academic progress than 
students who are offered their colleges’ standard services? Across the six colleges, do the 
programs’ impacts vary by any particular subgroups of students? Do the programs’ impacts 
vary by college or by cohort of entry into the program within each college? 
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Chapter 4 reviews what is known about the implementation of the programs at the six 
colleges to inform the interpretation of the programs’ impacts. The chapter considers the 
strength of the program models, the fidelity and variation in their implementation (both within 
and between colleges), and the contrast between the experiences of students in the program and 
control groups. The chapter also offers some possible explanations for why the estimated 
impact for one outcome of one program, the Opening Doors learning communities at Kings- 
borough, was larger than that of the other programs. 

The costs of offering learning communities for students in developmental courses are 
described in Chapter 5. This chapter looks most closely at program costs at CCBC, Houston, 
and Kingsborough, where detailed cost data were collected, but also at common elements and 
the average costs of educating students across the six colleges included in this analysis to put 
these findings in context. The chapter also provides an analysis of these costs in relation to the 
average observed effects of learning communities. 

Chapter 6 draws summary conclusions and discusses the implications of these findings 
for the role of learning communities in serving community college students who are in need of 
developmental education. 
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Chapter 2 

Student Characteristics and Study Design 


The six colleges discussed in this report — the Community College of Baltimore County 
(CCBC), Hillsborough Community College, Houston Community College, Kingsborough 
Community College, Merced College, and Queensborough Community College — represent a 
variety of geographic locations in urban and suburban areas across the country and implemented 
a variety of learning community models. The colleges and their programs were described in 
Chapter 1. The student population from which the students in the study were recruited varied 
somewhat across the six colleges, although all students had tested into developmental education 
in either English or mathematics or both. This chapter begins with a description of the eligibility 
criteria for students to participate in the study, followed by a summary of the characteristics of 
the study’s sample members. Finally, the design of the study is explained. 1 

Eligibility Criteria 

At five of the six colleges described in this report, the learning community programs targeted 
students in need of developmental education, rather than the colleges’ entire student body. Only 
at Kingsborough were “college-ready” students also included in the eligible population. This 
synthesis focuses on the effectiveness of learning communities that include a developmental 
course as part of the learning community; consequently, in this report Kingsborough’ s sample 
includes only those students with developmental English needs. For details on the specific 
eligibility requirements of each study, see Table 2.1. 

Across most of the colleges, students needed to meet an age requirement to participate 
in the study (usually 17 or 18 and older) and had to be available to enroll in a learning commu- 
nity at its scheduled time. In addition, some colleges recruited freshmen or returning students, 
while others focused on freshmen only. Lastly, students were eligible only if they were in need 
of developmental education in the subject area targeted by the college (for example, Merced 
targeted students in need of developmental English, and Houston targeted students in need of 
developmental math). Again, the original Kingsborough sample included students with and 
without developmental needs; in this report, only those students with developmental needs in 
English are included in the analyses. 


'The information provided in this chapter is intentionally brief. For more details, see the college-specific 
reports: Scrivener et al. (2008); Weiss, Visher, and Wathington (2010); Weissman et al. (2011); Weissman et 
al. (2012). 
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The Learning Communities Demonstration 
Table 2.1 

Study Eligibility Criteria, by College 


Final Report of the Learning Communities Demonstration 


College 

Eligibility Criteria 

The Community College of Baltimore 
County (CCBC) a 
(Baltimore, MD) 

• Placement into higher of two levels of developmental English 

Hillsborough Community College 
(Tampa, FL) 

• Placement into either of two levels of developmental reading 

• First-time student 

• Age 1 8 or over 

Houston Community College 
(Houston, TX) 

• Placement into lowest of three levels of developmental math 

• First-time student or transfer student with fewer than 1 2 credits 

• Age 17 or over 3 

Kingsborough Community College 
(Brooklyn, NY) 

• Placement into any of three levels of developmental English 13 

• First-time student 

• Planning to attend college full time 

• Age 1 7 to 34 c 

Merced College 
(Merced, CA) 

• Placement into any of the highest three (of four) levels of 
developmental English 

• Age 17 or over 3 

Queensborough Community College 
(Queens, NY) 

• Placement into either of two levels of developmental math 

• First-time or returning student with 1 5 or fewer credits 

• Age 17 or over 3 


SOURCES: Scrivener et al. (2008); Weiss, Visher, and Wathington (2010); Weissman et al. (2011); Weissman et 
al. (2012). 


NOTES: a At CCBC, Houston, Merced, and Queensborough, only students age 18 and over were initially eligible 
for learning communities. CCBC later eliminated this age requirement, and Houston, Merced, and Queensborough 
changed it to age 17 or over; students under 18 at all sites received parental consent to participate. 

b At Kingsborough, students who placed into college-level English were also eligible to participate in the 
study. Since this report focuses on developmental education learning communities, these students are excluded 
from the analysis and discussion. Analysis of the full sample of students in Kingsborough's Opening Doors study 
can be found in Bloom and Sommo (2005), Scrivener et al. (2008), and Sommo, Mayer, Rudd, and Cullinan 
( 2012 ). 

c During the first semester of study intake at Kingsborough, only students age 18 to 34 with a household 
income under 250% of the federal poverty level were eligible; in subsequent semesters, the income criterion was 
removed and 17-year-olds were eligible with parental consent. 
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Selected Characteristics of Sample Members at Baseline 

Table 2.2 provides background characteristics of sample members at baseline (that is, when 
they first enrolled in the study). Mirroring national trends and each college’s student population 
(as shown in Table 1.2), the majority of study participants at each college were women. Study 
participants were primarily of traditional college age at baseline. Both between and within the 
colleges, study participants were racially diverse. In three of the six colleges (Hillsborough, 
Kingsborough, and Queensborough), there was no racial majority. At Merced and Houston, the 
majority of study participants were Hispanic, and at CCBC the majority were black. None of 
these groups represented more than 60 percent of study participants at their college — all the 
colleges were racially heterogeneous. Most study participants did not have any children at the 
start of the study, although at Houston and Merced 28 percent and 26 percent had children, 
respectively. Between one-quarter and one-half of students knew that they would be receiving 
financial aid during the semester of random assignment, but over one-quarter of students did not 
know if they would be receiving financial aid. Most students in the sample had a high school 
diploma, although at Kingsborough 31 percent of the sample had a General Educational 
Development certificate (GED). 

Like community colleges nationally, a sizable proportion of study participants were the 
first person in their family to attend college, including 40 percent of study participants at 
Houston and 36 percent at Merced. Finally, between 40 and 50 percent of study participants at 
Houston, Kingsborough, and Merced reported that a language other than English was spoken 
regularly in their home. As Table 2.2 shows, study participants came from a diverse set of 
backgrounds and experiences. 

Data Sources 

The evaluation of learning communities for developmental students relies on several data 
sources. These data sources include: 

The Baseline Information Form 

Just before being randomly assigned, all students completed a short survey called the 
Baseline Information Form. This form collected information on demographic and other back- 
ground characteristics. Baseline data are used to describe the sample, to examine the similarity 
between research groups at the onset of the study, and to identify students for subgroup anal- 
yses. 
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SOURCE: MDRC calculations from Baseline Information Form data. 
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Operational Site Visits, Field Research, and Instructor Survey 

Implementation research was conducted at all six sites, and a variety of methods were 
used to collect data. Field research visits were conducted at least twice at all but one college, 
where only one visit was made. During these trips, the research team interviewed college 
administrators, faculty, and staff involved in the learning communities program. The interviews 
provided information on the operation of the program and included questions about key differ- 
ences between the program and the college’s standard services (what the control group was 
offered). The research team conducted informal observations of some learning community 
classes. Groups of students were interviewed to gain a deeper understanding of their experienc- 
es at the college and, for program group students, in the learning community program. 

Additionally, a survey of instructors was administered to document the characteristics 
and pedagogical beliefs and practices of learning communities instructors as well as some 
instructors who taught in stand-alone versions of those courses that were linked in the learning 
communities (that is, instructors whom control group students may have encountered if they 
signed up for courses similar to the learning communities linked courses). Survey questions 
were designed to capture the use of instructional strategies commonly associated with learning 
communities; participation in professional development opportunities; and teacher characteris- 
tics that might be associated with differences in teaching approaches, such as age, gender, 
seniority, and part-time versus full-time status. More information about the instructor survey is 
provided in Appendix C. 

These data sources were used primarily to describe the learning communities programs 
and how they were implemented, to illustrate how the programs were different from the 
colleges’ standard services, and to describe the evolution of the programs over time. Site- 
specific findings from the implementation research can be found in previously published reports 
(see the list at the end of this report). In this report, the findings are synthesized across all six 
sites to discern patterns that may shed light on the results of the impacts analyses reported in 
Chapter 3. 

Student Records 

Each college provided information on program and control group students’ academic 
outcomes from their student transcripts. These data are used to provide a detailed look at sample 
members’ academic progress on outcomes like enrollment and credit accumulation. This report 
presents a range of transcript data outcomes during the “program” semester, when program 
group members were eligible to participate in learning communities, and in two subsequent 
“postprogram” semesters. 
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Cost Data 

Three colleges (CCBC, Houston, and Kingsborough) provided budget data on their re- 
spective learning community programs; these documents are the primary source for program 
cost estimates. See the college-specific reports for details. 2 To estimate control group costs, 
operating budgets and annual credit hours attempted for all six colleges were pulled from the 
Integrated Postsecondary Education Data System (IPEDS). IPEDS is a system of interrelated 
surveys conducted annually by the U.S. Department of Education’s National Center for Educa- 
tion Statistics that gathers information from every college, university, and technical and voca- 
tional institution that participates in federal student financial aid programs. 

Methods 

Random Assignment Process 

College staff invited eligible students to participate in the learning communities evalua- 
tion through letters, e-mails, and phone calls, as well as in person during the registration 
process. As described above, students who attended an intake session on campus and agreed to 
take part in the study completed an Informed Consent Form and a Baseline Information Form 
containing questions about their background characteristics. After completing these forms, each 
student was assigned, at random, either to the program group, whose members have the oppor- 
tunity to participate in learning communities, or to the control group, whose members receive 
the college’s standard services. 3 Across the six colleges, on average, around 71 percent of 
students assigned to the learning communities were enrolled in a learning community at the 
add/drop deadline of the first semester of study. Less than 1 percent of control group members 
were enrolled in a learning community at the add/drop deadline of the first semester of study. 

Each college’s research sample was comprised of three or four groups (or cohorts) of 
students. Each cohort started at the beginning of subsequent semesters. For example, at Hills- 
borough, students in the first cohort were randomly assigned during the registration period for 
the fall 2007 semester, and the program group enrolled in learning communities in fall 2007 ; the 
second cohort of students did so in spring 2008, and the third in fall 2008. The random assign- 
ment process occurred separately at each college for each cohort. 

A Note on Random Assignment 

The deliberate random assignment process yields two groups of students that are similar 
both in characteristics that can be measured, such as age and gender, and those that are more 

2 Weissman et al. (201 1); Weissman et al. (2012); Sommo, Mayer, Rudd, and Cullman (2012). 

3 The random assignment decision was controlled by MDRC using a computer algorithm. 
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difficult to measure, such as tenacity, ability, and motivation. As a result, subsequent substantial 
differences in outcomes can be causally attributed, with a high level of confidence, to systemat- 
ic differences in students’ experiences after they were randomly assigned — in this case, the 
opportunity to experience the colleges’ learning communities. 

A random assignment evaluation is an extremely reliable way to test a program’s aver- 
age overall effectiveness; however, there are limitations to this method. Random assignment 
does not typically allow researchers to separate the effects of one program component from 
another. For the Hillsborough learning communities program, for example, this study deter- 
mined whether the entire program package was effective. This package included the linking of 
two classes (creating cohorts of students), a college success course (focusing on acclimation to 
college life and study skills), certain instructional strategies (such as integration of material 
across the two courses), and the qualities of teachers who taught in the learning communities . 4 
The qualitative research conducted as part of these studies provides information on which 
components of the program packages mattered most to the program leaders, faculty, and 
students who participated in the learning communities; however, it cannot answer the question 
of which components mattered most for student outcomes such as credit accumulation and 
persistence to the next semester. 


4 See Weiss (2010) for a discussion of the role of teacher effects in individually randomized experiments. 
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Chapter 3 

Effects on Academic Progress 


This chapter presents findings on the average effectiveness of learning communities during the 
program semester and two subsequent postprogram semesters. The focus is on whether the 
opportunity to participate in learning communities enabled students to make greater academic 
progress than they would have made had they been offered the colleges’ usual services. 1 
Findings are first pooled across six colleges and then pulled apart to understand how the 
programs’ effects vary across colleges, cohorts, and different types of students. 2 

As described in detail in Box 3.1, the programs’ pooled impact or effect is estimated by 
comparing the outcomes of all students who were randomly assigned to the program group with 
those of the control group. Differences between the two groups represent the estimated value 
added of the opportunity to participate in learning communities, above and beyond the oppor- 
tunity to receive the colleges’ usual services. 3 

Briefly, the key findings show that learning communities had: 

• No discernible effect on persistence. Students in the program group were 
no more likely than the control group to enroll in college in the first, second, 
or third semester after they entered the study. 

• A positive effect on progress in a targeted subject (either English or 
mathematics). Learning communities, on average, had a positive impact on 
students’ attempting and earning credits in a targeted subject, either English 
or mathematics (driven by developmental credits earned). The program’s 
half-credit impact in the program semester was maintained for up to two se- 
mesters after the program. 


'Analyses in this chapter reflect the effect of the opportunity to participate in learning communities, which 
is not necessarily the same as the effect of participation in learning communities. Some individuals assigned to 
the program group did not participate in learning communities; however, they are included in the program 
group for the analyses in order to ensure a comparison of two groups that were similar at the outset of the 
study. For simplicity of presentation the phrase “the opportunity to participate in. . .” will typically be dropped, 
but readers should be aware that analyses reflect the opportunity unless otherwise stated. 

“As described in Chapter 1, although the Learning Communities Demonstration included six colleges, 
only the five colleges that ran developmental education learning communities are included in the main pooled 
analyses. The sixth site included in the main pooled analyses is the Opening Doors learning communities 
program at Kingsborough Community College, which (for the majority of students) ran developmental 
education learning communities. 

3 Additional information relevant to the impact analyses presented in this report is provided in Appendix A. 


29 



Box 3.1 


How to Read the Impact Tables in This Report 

Most tables in this report use the format illustrated in the abbreviated table below, which 
displays some hypothetical transcript data for the program and control groups. The first row 
shows that program group students earned an average of 9. 1 credits and control group students 
earned an average of 6. 1 credits. 

The “Difference” column in the table shows the observed difference between the two research 
groups on the outcome — that is, the estimated average impact of the opportunity to partici- 
pate in the program. For example, the estimated average impact on credits earned can be 
calculated by subtracting 6.1 from 9.1, yielding an impact estimate of 3.0 credits earned. This 
difference represents the estimated average impact rather than the true average impact (which 
is impossible to determine) because, although study participants are randomly assigned to the 
program and control groups, the impact estimate would have been different if a different 
sample of students had been included in the study or if the same group of students had been 
randomized in a different way. 

Differences marked with one asterisk or more are considered statistically significant, meaning 
that there is a high probability that the opportunity to participate in the program had an impact 
on that outcome measure. Differences that have no asterisk indicate that the opportunity to 
participate in the program did not have a discernible effect on that outcome. The number of 
asterisks indicates the probability that an impact at least as large as the one observed in the 
study would have occurred even if the true average impact had been zero. One asterisk corre- 
sponds to a 10 percent probability; two asterisks, a 5 percent probability; and three asterisks, a 
1 percent probability. The more asterisks, the more likely the opportunity to participate in the 
program had a true average impact on the outcome. The impact in the table excerpt below has 
three asterisks, indicating that the impact is statistically significant at the 1 percent level — 
meaning that there is only a 1 percent chance of observing an estimated average impact this 
large (or larger) if the opportunity to participate in the program actually had no average effect 
on credits earned. In other words, there is a 99 percent level of confidence that the opportunity 
to participate in the program had a positive impact on the average number of credits earned. 

Also shown in the table is the standard error of the impact estimate. The standard error is a 
measure of uncertainty or variability around the impact estimate. Some useful rules of thumb 
are that there is about a 90 percent chance that the true average impact is within plus or minus 
1.65 standard errors of the estimated average impact, roughly a 95 percent chance that the true 
average impact is within plus or minus 1.96 standard errors of the estimated average impact, 
and about a 99 percent chance that the true average impact is within plus or minus 2.58 stand- 
ard errors of the estimated average impact. For example, in the data below, there is roughly a 
99 percent chance that the average impact on credits earned lies between 0.9 and 5.1, calculat- 
ed as 3.0 ± (2.58 x 0.8). 



Program 

Control 

Difference 

Standard 

Outcome 

Group 

Group 

(Impact) 

Error 

Credits earned 

9.1 

6.1 

3 Q *** 

0.8 
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• No discernible effect on progress outside the targeted subject. Learning 
communities had no discernible effect on students’ credit accumulation out- 
side the targeted subject (primarily college-level credits). 

• A small positive effect on overall academic progress (total credits 
earned). During the program semester, learning communities students 
earned half a credit more than their control group counterparts, representing 
an 8 percent increase in total credit accumulation. This was a result of stu- 
dents earning half a credit more in the targeted subject. Over the following 
two postprogram semesters, the cumulative estimated impact remained the 
same (half a credit), although it was no longer statistically significant by the 
third semester. 

• Varying effectiveness among colleges with respect to progress in the tar- 
geted subject area but fairly similar effects across colleges with respect 

to overall academic progress. The programs’ average effects varied across 
the six colleges with respect to credits earned in the targeted subject area. 
However, the programs’ average effects were fairly similar across the col- 
leges with respect to total credit accumulation, the best proxy for overall pro- 
gress toward a degree. This suggests that the pooled results are a reasonable 
summary of the average effectiveness of learning communities at these six 
colleges. This does not preclude the possibility that the effects of learning 
communities vary within colleges (that is, between learning community links 
or between teaching teams at each college). 

The remainder of this chapter is organized into several sections. First, the research ques- 
tions addressed in the impact analyses are explained. Then, a detailed description is provided of 
the main outcomes used to measure the program’s effectiveness. Next, the pooled effects of 
learning communities across six colleges are presented, followed by analyses of whether the 
effects vary across colleges, subgroups of students, and different cohorts of students. Finally, 
conclusions are offered. 

Research Questions 

Broadly speaking, the main impact analyses seek to answer the following question: 

• Did students who were offered the opportunity to participate in learning 
communities programs make better academic progress than students who 
were offered their colleges’ standard services? 
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To answer this question, the programs’ pooled impacts on three key indicators of aca- 
demic progress are examined (for detailed descriptions of how these indicators were measured, 
see the next section of this report): 

1. Progress in the targeted subject area (English or mathematics) 

2. Progress outside the targeted subject area 

3. Overall progress toward a degree 

These indicators are measured cumulatively at the end of each of the first three semes- 
ters after students entered the study to determine learning communities’ effectiveness during the 
program semester and whether any initial positive effects are maintained in two subsequent 
postprogram semesters. 

In addition to these main analyses, additional research questions include: 

• Was the opportunity to participate in learning communities more or less ef- 
fective for particular subgroups of students? 

That is, does the effectiveness of learning communities vary by students’ characteris- 
tics? In addition: 

• Did the effect of the opportunity to participate in learning communities vary 
by college and/or cohort? 

With six colleges and three to four cohorts of students at each college, this study pro- 
vides a unique opportunity to explore whether average program impacts varied across colleges 
and cohorts. Separate unbiased impact estimates can be obtained for each college by cohort, and 
an assessment can be made as to whether the amount of observed variation in impacts is greater 
than what is expected if the programs’ true effects were homogenous across colleges and/or 
cohorts. 

Description of Key Outcomes 

Before delving into the findings, it is important to understand the outcome measures used 
to assess the programs’ effectiveness and why they were selected. The three key outcomes are: 

1 . Progress in the T argeted Subject Area 

The learning community programs all included at least one subject-specific develop- 
mental course in their learning community link. This targeted subject area varied from college 
to college. The Community College of Baltimore County (CCBC), Hillsborough Community 
College, Kingsborough Community College, and Merced College targeted developmental 
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English (reading and/or writing). Houston Community College and Queensborough Communi- 
ty College focused on developmental mathematics. While the colleges targeted different subject 
areas, college staff expected that learning communities would help students progress through 
the targeted subject’s course sequence more quickly. Faster progression could occur by getting a 
greater proportion of students to attempt (and pass) courses and/or by increasing pass rates 
among students who attempt courses. 

To assess the program’s success on this outcome, the number of credits students earn in 
the targeted subject is measured. For example, at those schools targeting developmental math, 
this measure sums the total number of math credits earned, including both developmental and 
college-level credits. During the first semester of study, this outcome represents primarily 
credits earned in the targeted subject area’s developmental course that is part of the learning 
community link . 4 This measure does not include any credits earned outside the targeted subject 
area. For example, Houston linked developmental mathematics with a student success course; 
this measure includes only credits earned in mathematics and does not include credits earned in 
the student success course. 

Using the measure “credits earned in the targeted subject area” as an indicator of pro- 
gress has the advantage of being meaningful in the short and medium terms, marginally by 
semester, and cumulatively over time. In addition, this measure is meaningful regardless of 
students’ initial level of need in the targeted subject area, which is important because students 
started at various levels of developmental need both within and between colleges . 5 

2. Progress Outside the Targeted Subject Area 

As noted, a short-term goal of the studied learning communities was to help students 
progress through a targeted subject’s developmental course sequence more quickly. Focusing 
efforts on a specific subject’s developmental course sequence may lead to improved progress in 
courses outside that subject area. For example, a student with developmental reading needs may 
have an increased chance of succeeding in her nonreading courses if she completes develop- 
mental reading earlier in her schooling, since the skills learned may be necessary to succeed in 
other courses, like history. In addition, the learning communities theory of change suggests that 

4 During the first semester of study, 98 percent of credits earned in the targeted subject area were develop- 
mental credits. 

5 Other indicators of progress were considered, but each was subject to the constraints described in the text. 
For example, completion of the targeted sequence was considered, since it may be considered an important 
benchmark in a students’ academic progress. However, a binary indicator like “completion” would fail to credit 
learning communities for moving students from lower levels of developmental education to higher levels of 
developmental education. This is particularly problematic in earlier semesters of study, when certain students 
(those who began the study multiple levels below college-level courses) were extremely unlikely to complete 
the sequence during the program semester. 
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students in learning communities learn critical thinking, problem solving, and higher-order 
cognitive skills because of the integration of their course work across classes. This could enable 
them to perform better in courses beyond the targeted math or reading course. Measuring 
progress through courses outside the targeted subject area has the potential to capture positive 
spillover effects. 

While completing required subject area developmental courses earlier may be advanta- 
geous, it is unclear whether it is helpful if students achieve this goal by (a) putting off other 
required courses (substituting the targeted subject’s developmental course for a course they 
otherwise would have taken), or (b) focusing their time and energy on the targeted subject to the 
detriment of performance in other subject areas. Ignoring the possibility of a “substitution 
effect” or declined (or improved) performance in other subject areas is akin to conducting a 
clinical trial without examining negative (or positive) side effects. Measuring progress through 
all courses outside the targeted subject area has the potential to capture these side effects. 

As a result, the second main measure of academic progress is credit accumulation out- 
side the targeted subject area, including both developmental and college-level credits. Again, 
these credits include the college-level credits earned in both college-level courses such as 
sociology and biology as well as credits earned in the “student success” courses that were part 
of several of the colleges’ learning communities links. In the first semester after students were 
randomly assigned, 78 percent of credits earned outside the targeted subject area were college- 
level credits. 

3. Overall Progress Toward a Degree 

Ultimately, the most important question this report addresses is whether learning com- 
munities improve students’ overall progress toward a degree. There is no perfect measure of 
“overall progress toward a degree,” only useful proxies. This report’s proxy is calculated by 
summing together all credits earned inside and outside the targeted subject area, including 
developmental and college-level credits. In order for students to earn an associate’s degree, they 
must complete at least 60 college-level credits. For students who place into developmental 
course work, developmental courses may be required before taking certain required college- 
level courses, so these students essentially must complete more than 60 credits in order to earn a 
degree. 6 


6 Across the six colleges in this study, the targeted developmental education course was required at three 
colleges (CCBC, Houston, and Kingsborough). At Hillsborough, the course was a prerequisite for many other 
courses. At Queensborough, the targeted developmental course was required for enrollment in college-level 
courses in that subject. At Merced, the targeted developmental course was not strictly required, but students 
were expected to complete it before moving on to college-level courses in that subject. 
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There are limitations to using total credits earned as a proxy for progress toward a de- 
gree — for example, not all combinations of 60 college-level credits result in a degree, since 
there are different requirements for different majors. Different combinations of the same 
number of total credits earned may actually suggest that a student is closer to or further from 
earning a degree. In addition, some students skip the developmental course they place into, and 
since developmental credits do not count toward a degree, skipping developmental courses may 
bring students closer to a degree, even though they have earned fewer credits. Random assign- 
ment ensures that at the start of the study, program and control group students had similar levels 
of developmental need, but it is possible that the programs could influence students’ likelihood 
of skipping developmental courses. With those caveats in mind, in general, earning more 
developmental and college-level credits brings a student closer to earning a degree, so total 
credit accumulation serves as a good indicator of overall progress toward a degree. 

One limitation of this study’s outcome measures is that none of the progress measures 
presented in this report attempt to measure student learning directly. This is a common limita- 
tion in community college research, since there are not standardized assessments of students’ 
learning offered among colleges, and frequently there are not standardized assessments of 
students’ learning within colleges. 7 Nevertheless, since college degrees and certificates are 
awarded based on credit accumulation, the academic progress indicators used in this study are 
outcomes that are important to college administrators, policymakers, and researchers. 

Pooled Analyses 

The remainder of this chapter presents the impact findings. Presented first are the pooled effects 
of the learning communities programs operated at six colleges, including nearly 7,000 students 
and 174 learning communities. The pooled analysis provides an estimate of the overall average 
effect of the opportunity to participate in the studied learning communities, which targeted 
students with developmental needs. Following the overall findings, results are unpacked to 
better understand what went into the pooled impact estimates. 

Table 3.1 presents the pooled average effects of learning communities across the six 
studied colleges. 


7 In addition, even if standardized measures were available, it would be extremely costly, difficult, and 
likely unreliable to attempt to measure academic achievement for the many students who no longer attend the 
institution at which they began the study. Although Grade Point Average (GPA) is not a standardized measure 
of learning, it is sometimes used as a proxy for learning. Since GPA cannot be calculated for students who do 
not enroll, the use of GPA as an outcome measure can be complicated in an experiment. Interpretation of GPA 
impacts is further complicated when students take courses of different difficulty. 
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Impact on Enrollment in College 

One key mechanism by which learning communities are theorized to improve students’ 
progress toward a degree is by increasing their likelihood of enrolling in subsequent semesters 
after the program ends. Students in learning communities are expected to benefit from the peer 
support that being part of a cohort is intended to foster. Faculty collaboration around a shared 
set of students may lead to more personal relationships with students, creating a sense of 
belonging. Teaching in learning communities classes is supposed to be more engaging than in 
the regular classroom. A consequence of these multiple factors, according to the learning 
communities’ theory of change, is that students in learning communities will be more likely to 
persist in college. Although enrollment is not listed as a key outcome in this study, it is very 
important to examine as it relates to the theory of change. As shown in Table 3.1, this study 
finds that the studied one-semester learning communities have no discernible effect on enroll- 
ment during the two semesters after students participated in the program. 

While enrollment is an important outcome with respect to the learning communities 
theory of change, enrollment without credit accumulation is of limited value. In contrast, 
impacts on credit accumulation without impacts on enrollment are still extremely important. As 
a result, this report focuses on credit accumulation, treating enrollment as a secondary, mediat- 
ing outcome. Examined next are the three primary outcomes in this study. 

1 . Impact on Credits Earned in the Targeted Subject Area 

Learning communities had a positive effect on credit accumulation in the targeted sub- 
ject area. As shown in Table 3.1, during the semester in which program group students first 
participated in a learning community, they earned an average of 2.72 credits in the targeted 
subject, compared with 2.20 credits for control group students. In the program semester, these 
credits are almost entirely developmental. The 0.52 credit impact represents a 24 percent 
increase over the control group average (0.52 4 - 2.20). For context, most developmental courses 
in this study carried three or four credits, so the estimated impact represents around one-sixth or 
one-eighth of a typical developmental course. 

It is important to understand how this fairly modest impact came about. Was the pro- 
grams’ positive effect on average credits earned in the targeted subject a result of students 
attempting more credits in that subject, or passing those courses at a higher rate? 

As shown in Table 3.1, program group students attempted an average of 4.13 credits in 
the targeted subject, whereas control group students attempted an average of 3.57 credits in the 
targeted subject. Thus, the program got students to attempt an average of 0.56 more credits in 
the targeted subject. The fact that the estimated impact on credits attempted in the targeted 
subject (0.56 credits) lines up well with the estimated impact on credits earned in the targeted 
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The Learning Communities Demonstration 
Table 3.1 

Pooled Academic Outcomes 

Final Report of the Learning Communities Demonstration 


Outcome 

Program 

Group 

Control 

Group 

Difference 

(Impact) 

Standard 

Error 

Enrollment ( % ) 

Enrollment rate a 

First semester 

84.1 

82.9 

1.2 

1.2 

Second semester 

63.8 

62.5 

1.3 

1.3 

Third semester 

49.9 

50.2 

-0.3 

1.4 

Full-time registration 1 

First semester 

54.9 

52.0 

29** 

1.4 

Second semester 

39.4 

39.1 

0.3 

1.2 

Third semester 

29.4 

29.5 

-0.1 

1.2 

Enrolled in a learning community 

First semester 

70.8 

0.7 

70 \ *** 

1.6 

In the targeted subiect c 

Cumulative credits attempted 

First semester 

4.13 

3.57 

0.56 *** 

0.13 

Second semester 

5.83 

5.23 

0.60 *** 

0.17 

Third semester 

6.72 

6.19 

0.53 *** 

0.19 

Cumulative credits earned 

First semester 

2.72 

2.20 

0.52 *** 

0.12 

Second semester 

4.02 

3.52 

0.50 *** 

0.16 

Third semester 

4.69 

4.25 

0.44 ** 

0.17 

Completed targeted developmental course sequence (%) 

Third semester 

37.6 

35.0 

2.5 

1.5 

Outside the targeted subiect 

Cumulative credits attempted 

First semester 

6.04 

6.31 

-0.27 ** 

0.13 

Second semester 

11.14 

11.37 

-0.23 

0.23 

Third semester 

15.44 

15.60 

-0.17 

0.34 

Cumulative credits earned 

First semester 

4.19 

4.19 

0.00 

0.12 

Second semester 

7.88 

7.84 

0.05 

0.22 

Third semester 

10.87 

10.85 

0.01 

0.30 


(continued) 
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Table 3.1 (continued) 


Outcome 

Program 

Group 

Control 

Group 

Difference 

(Impact) 

Standard 

Error 

Total credits 

Cumulative credits attempted 

First semester 

10.18 

9.88 

0.29 * 

0.18 

Second semester 

16.99 

16.60 

0.38 

0.31 

Third semester 

22.18 

21.81 

0.37 

0.43 

Cumulative credits earned 

First semester 

6.91 

6.38 

0.53 *** 

0.17 

Second semester 

11.92 

11.36 

0.56 * 

0.30 

Third semester 

15.58 

15.10 

0.47 

0.39 

Sample size (total = 6,974) 

3,983 

2,991 




SOURCE: MDRC calculations from the Community College of Baltimore County, Hillsborough Community 
College, Houston Community College, Kingsborough Community College, Merced College, and Queensborough 
Community College transcript data. 

NOTES: Rounding may cause slight discrepancies in sums and differences. Targeted subject credit measures and 
outside the targeted subject credit measures may not sum to total credit measures because Hillsborough 
Community College includes vocational credits in total credit measures. These vocational credits are not included 
in targeted subject credit measures or in outside the targeted subject credit measures. Cumulative measures 
exclude repeated courses. 

A two-tailed t-test was applied to differences between research groups. Statistical significance levels are 
indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 

The probability of being assigned to the treatment group varies across colleges and within random assignment 
cohorts, and estimates are weighted to account for the different random assignment ratios. Estimates are adjusted 
by campus and cohort. Standard errors are clustered by learning community link. 

a Enrollment is based on courses that sample members are still enrolled in at the end of the add/drop period. 

b Students are registered full time if they attempt 12 or more credits in a semester. 

c Courses in the targeted subject for Kingsborough Community College, Hillsborough Community College, 
the Community College of Baltimore County, and Merced College are English courses and include both college- 
level and developmental credits. Courses in the targeted subject for Houston Community College and 
Queensborough Community College are math courses and include both college-level and developmental credits. 


subject (0.52 credits) might lead one to think that the effect on credits earned was solely a result 
of students attempting more credits. However, it is unlikely that this is true. While the program 
induced students to attempt 0.56 more credits in the targeted subject, the control group pass rate 
for the courses bearing these types of credits is only 62 percent (2.20 4- 3.57), so it is unlikely 
that nearly every additional credit attempted by program group members would translate into an 
additional credit earned. 

Assuming program group students earned all their attempted credits at the same rate as 
control group students (62 percent), the expected impact on credits earned that resulted from the 
additional credits attempted alone would have been 0.34 credits (0.62 x 0.56). This suggests that 
around 66 percent (0.34 4 0.52) of the actual estimated impact on credits earned was a result of 
students attempting more credits, and the remaining 34 percent of the impact was a result of the 
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increased pass rate. The assumptions required for this decomposition to be correct are strong; 
nonetheless, it provides some means of unpacking the impact on credits earned in the targeted 
sequence. It suggests that this impact was mostly a result of students attempting more credits 
and partially a result of improved pass rates. This is important, because there are likely other, 
more direct ways that a college could influence students’ course-taking patterns. 

Students’ outcomes on credits earned in the targeted subject were tracked for an addi- 
tional year after the program was completed to assess whether the impact grew, decreased, or 
remained the same. The effect is maintained up to two semesters after program group students 
are no longer in learning communities. In other words, learning communities help students 
make some progress through a particular, targeted subject area. This progress occurs during the 
program semester, and the benefit is maintained through at least two postprogram semesters. 

Another way to consider learning communities’ effects in the targeted subject is to ex- 
amine completion of the developmental course sequence in that targeted subject area. 8 By the 
end of the third semester of study, 37.6 percent of program group students completed their 
developmental requirements in the targeted subject area, compared with 35.0 percent of control 
group students. The difference is right on the border of statistical significance (p = .104). The 
lack of discernible impacts on completion of the targeted subject sequence does not take away 
from the fact that learning communities helped some students make progress in the targeted 
subject. 


Learning communities’ positive effect on progress in the targeted subject is modest. It is 
important to keep in mind that this outcome is primarily of interest because of the hypothesis 
that attempting and earning credits in a targeted developmental subject early on will be benefi- 
cial to students’ overall progress in the longer term. Consequently, it is important to consider 
this result in tandem with students’ overall academic progress. 

2. Impact on Credits Earned Outside the Targeted Subject Area 

The learning communities programs had, on average, a small positive effect on stu- 
dents’ progress in a targeted subject. To properly interpret this effect, it is important to under- 
stand how students performed outside that targeted subject. Since students attempted more 
credits in the targeted subject, did they reduce their course load in nontargeted subjects? Since 
students earned more credits in the targeted subject, did this translate into their earning more or 
less credits in other subject areas? 


8 Since study participants started one, two, or three courses below the college-level course in the targeted 
subject area, many could not complete their developmental requirements during the first semester of study. 
However, by the end of the third semester all students, in theory, could have completed their developmental 
courses in the targeted subject area. 
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During the program semester, learning communities students attempted fewer credits 
(-0.27) outside the targeted subject than their control group counterparts. This suggests a small 
amount of “substitution” in students’ course enrollments. That is, learning communities induced 
more students to attempt credits in a particular targeted subject area; however, in doing so, on 
average, students attempted slightly fewer credits in other nontargeted subjects. Despite this 
substitution in attempted credits, the average number of credits earned outside the targeted 
subject was strikingly similar between program and control group students during the program 
semester. This occurred because program group students passed their untargeted credits at a 
slightly higher rate than control group students. 9 This implies that the substitution did not hold 
students back in their academic progress outside the targeted subject. Notably, while most of the 
credits in the targeted subject were developmental credits, most of the credits outside the 
targeted subject were college- level credits. 

After the program semester, program and control group students continued to attempt 
and earn credits outside the targeted subject at a similar pace. Despite the small positive effect 
on students’ progress in a targeted subject, generally speaking, the learning communities 
programs had no discernible effect (negative or positive) on students’ progress outside the 
targeted subject. 

3. Impact on Total Credits Earned 

Summing together credits earned in the targeted and untargeted subjects yields stu- 
dents’ total credits earned. During the program semester, learning communities students earned, 
on average, 0.53 total credits more than their control group counterparts. In the absence of 
learning communities, students earned an average of 6.38 total credits, so this increase of 0.53 
credits represents an increase of around 8 percent. The impact on total credits earned is a result 
of students earning more credits in the targeted subject. 

Recall that suggestive evidence indicates that the programs’ positive effect on credits 
earned in the targeted subject was mostly attributable to program group students’ attempting 
more credits, and partially because they earned these credits at a higher rate. In contrast, 
suggestive evidence indicates that the impact on total credits earned is mostly a result of 
students earning the credits they attempted at a higher rate, and partially because they attempted 
more total credits. The reason for the flip is that the programs had a slightly negative effect on 
credits attempted outside the targeted subject area, and yet students in both groups earned 
around the same number of credits, owing to program group students’ higher rates of earning 
the credits they attempted. Since among all credits, most are outside the targeted subject area, 
when the two credit types are summed together it is observed that around 64 percent of the 

9 The comparison of pass rates is not experimental and a statistical test was not conducted. 
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programs’ impact on total credits earned is attributed to increased pass rates, and the remaining 
36 percent is a result of increased credits attempted. As before, this finding is speculative but 
does provide context for understanding the programs’ impact on total credits earned. 

Over the two postprogram semesters (subsequent to the learning communities semes- 
ter), the magnitude of the estimated impact on total credits earned remains fairly stable. In other 
words, the programs’ impact on total credits earned occurs during the program semester, and 
the estimated advantage is largely maintained through two postprogram semesters (although the 
cumulative effect estimate is no longer statistically significant at the end of three semesters). 10 

Impact on “College-Level” Credits 

For simplicity of presentation and to limit the number of ways credit accumulation is 
disaggregated, Table 3.1 does not break total credits into developmental credits and college- 
level credits. 11 Credits earned in the targeted subject are primarily developmental credits, and 
credits earned outside the targeted subject are primarily college-level credits. Given this, 
unsurprisingly, on average, the learning communities programs have no discernible effect on 
college-level credits earned. Appendix Table B.l breaks total credits into developmental and 
college-level credits. 

Summary of Pooled Analyses 

In sum, during the program semester, students in learning communities earned half a 
credit more in the targeted subject area than their control group counterparts. This benefit does 
not come at the detriment of progress outside the targeted subject area, but it does not help 
students move through their nontargeted courses either. After the program semester is complete, 
learning communities students maintain the edge they gained during the program semester, but 
otherwise their academic trajectory remains the same. After three semesters, students enrolled in 
learning communities, on average, are one-half of a developmental credit closer to a degree than 
students who were offered their colleges’ regular services. 


l0 Even though the point estimate remains fairly constant from the first through the third semesters, it goes 
from being statistically significantly different from zero to not discernibly different from zero because the level 
of precision in the estimate gets worse over time, as reflected by the increasing standard error. This occurs 
because the variance of cumulative credits earned increases from semester to semester. 

1 'Developmental credits do not count toward a degree, although many schools require or strongly encour- 
age students to take developmental credits before advancing to certain college-level courses. 
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Impact Variation 

The pooled average impact of learning communities across multiple colleges (described above) 
provides a general estimate of the average effectiveness of the studied programs. However, 
overall average impacts can mask differences in impacts across colleges and types of students. It 
is therefore helpful to formally test whether the learning communities programs’ effects are 
heterogeneous. Policymakers and other decision makers have become increasingly interested 
not only in understanding what works, but also for whom and under what circumstances. A 
program whose impacts vary substantially and unpredictably may be a risky option for decision 
makers. A program whose impacts vary predictably may have implications for targeting 
services to identifiable subpopulations. 12 

This section of the report seeks to understand whether the learning communities’ effec- 
tiveness varied across the six studied colleges, whether the programs’ effects varied across a 
few key subgroups of individuals, and whether there is evidence that the programs’ effective- 
ness varied across the different cohorts of students in the study. 

Impact Variation by College 

College-by-college impact findings are presented in detail in Appendix Tables B.2 
through B.7. 13 Since the random assignment of students occurred within each college, it is 
possible to obtain an experimental (unbiased) estimate of the effect of learning communities at 
each college. 

Comparing the impact findings among colleges requires caution — it is not possible to 
pin down the exact factors that cause variation in impacts among colleges. Differences in 
impacts among colleges may be a result of a variety of factors, including the students’ charac- 
teristics, the context, and the treatment contrast. 

• Student characteristics refers to the background characteristics of the stu- 
dents served at the colleges, such as students’ gender or initial achievement. 

For example, if learning communities are more effective for women than for 
men, and the colleges serve different proportions of men than women, then 
the programs’ effects could vary across colleges. 

• Context refers to any broader environmental factors, such as local job pro- 
spects. For example, if local job prospects vary among college contexts, and 

l2 Bloom, Raudenbush, and Weiss (forthcoming). 

13 Additional college-specific analyses and discussion are also available in previously released reports: 
Scrivener et al. (2008); Weiss, Visher, and Wathington (2010); Weissman et al. (2011); Weissman et al. 
(2012). 
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learning communities’ effects vary depending on students’ employment op- 
portunities, then the programs’ effects could vary across colleges. 

• Treatment contrast refers to the difference between the experiences of pro- 
gram group students at each college (program services received) and the ex- 
perience of control group students at each college (regular services received). 

An example of the treatment contrast could be differences in the type or in- 
tensity of support services program group students receive compared with 
control group students. Should the program group levels vary across colleg- 
es, or the program-control group differences vary across colleges, then the 
programs’ effects could vary across colleges. 14 

In addition to the above sources of variation in impacts, a certain amount of variation in 
observed impact estimates is expected due to chance. 15 The analyses below are intended to 
determine whether the observed variation in impacts among colleges is greater than what is 
expected due to chance. If there is not strong evidence that the impacts vary across colleges, 
then the pooled results described above may provide a reasonable description of the average 
effect of learning communities at the six colleges in this study. If there is evidence that the 
programs’ impacts vary across the six colleges, then there is good reason to believe that this 
type of program is more effective at some colleges and less effective at others, for the types of 
reasons described in the previous bullets. 

Figures 3.1 through 3.3 display the estimated impacts on the three key outcomes of in- 
terest, for each college (labeled with black squares), as well as pooled across all six colleges 
(labeled with a white diamond). In addition, these graphics include 95 percent confidence 
intervals around each impact estimate as a way of representing the uncertainty around each 
impact estimate. These figures are intended to provide a visual representation of variation in the 
impact of learning communities among the colleges on each outcome. The details provided in 
Appendix Tables B.2 through B.7 provide important context for interpreting these impacts, 
since those tables include control group levels and program participation rates, which vary 
among colleges, and additional outcomes like credits attempted, which may influence credits 
earned. 


14 This framework for the sources of variation in impacts is based on forthcoming work by Weiss and 
Bloom and has origins in the work of Raudenbush and Bryk (2002, p. 205). 

15 More formally, this is sometimes referred to as estimation error. 
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The Learning Communities Demonstration 
Figure 3.1 

Impact of the Learning Communities Program on Credits Earned in the 
Targeted Subject at the End of the Program Semester, by College 

Final Report of the Learning Communities Demonstration 
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SOURCE: MDRC calculations from the Community College of Baltimore County, Hillsborough Community 
College. Houston Community College, Kingsborough Community College, Merced College, and Queensborough 
Community College transcript data. 


Credits Earned in the Targeted Subject Area 

Figure 3.1 displays each college’s estimated impact and 95 percent confidence interval 
on the outcome progress through the targeted subject during the program semester. For ex- 
ample, at Houston the estimated impact on credits earned in the targeted subject is 0.43 credits. 
The confidence interval indicates 95 percent confidence that the true impact on this outcome 
lies between 0.22 and 0.64 credits. Confidence intervals that do not cross zero are considered 
statistically significant at the 5 percent level. 
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With respect to credits earned in the targeted subject, there is clear evidence that the ef- 
fect of learning communities varied by college. 16 While the pooled impact estimate is around 
0.52, there is more to the story than the pooled impact estimate. At Kingsborough, for example, 
the learning communities estimated impact is 1.49 credits, nearly one credit greater than the 
pooled average impact estimate; at other colleges (CCBC and Hillsborough), the estimated 
impacts are below the pooled average. This suggests that, with respect to credit accumulation in 
a targeted subject, learning communities have the potential to have a significantly larger impact 
than the pooled estimate, and they also can have a significantly smaller impact on this particular 
outcome. Signs of impact variation among colleges on cumulative credits earned in the targeted 
sequence remain evident through the end of the third semester of the study. 17 

Since there is evidence that the impacts vary by college, and the estimated impact at 
Kingsborough is one credit larger than the pooled average estimate, one might wonder what the 
pooled impact is when Kingsborough is excluded (that is, how do the findings for this outcome 
change when the largest impact is excluded)? With respect to credits earned in the targeted 
subject, excluding Kingsborough from the pooled analysis reduces the estimated impact in the 
program semester from 0.52 to 0.33 credits. 18 By the end of three semesters, the change in 
impact for this outcome goes from 0.44 to 0.22 credits. 19 Chapter 4 presents a discussion of the 
respects in which the program and setting at Kingsborough differed from that of the other 
colleges. The evaluations discussed in this report were not designed to disentangle the reasons 
for cross-college impact variation. That noted, across the six colleges there is a striking relation- 
ship between the programs’ estimated impacts on credits attempted and earned in the targeted 
subject (see Appendix Tables B.2 through B.7) — the programs with larger impacts on credits 
attempted have larger impacts on credits earned. 20 With only six colleges, it is difficult to 
establish a definitive pattern, but there is certainly suggestive evidence, which aligns well with 
the pooled analyses described above, that a key mechanism through which learning communi- 
ties produced impacts on credit accumulation in the targeted subject was by leading more 
students to attempt credits in that targeted subject. 21 


I6 A joint F-test yields a p-value less than .01. 

I7 A joint F-test yields a p-value equal to .04. 

l8 This impact would have been deemed statistically significant (p < .01 ). 
l9 This impact would not have been deemed statistically significant (p = .20). 

20 The correlation across the six colleges between the estimated impacts on credits attempted and credits 
earned in the targeted sequence is greater than .95. 

“'There is not a clear pattern between differences in the courses offered (for example, developmental math 
versus developmental English) in the learning communities’ links and the magnitude of the estimated impacts. 
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The Learning Communities Demonstration 
Figure 3.2 

Impact of the Learning Communities Program on Credits Earned Outside the 
Targeted Subject at the End of the Program Semester, by College 

Final Report of the Learning Communities Demonstration 
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SOURCE: MDRC calculations from the Community College of Baltimore County, Hillsborough Community 
College, Houston Community College, Kingsborough Community College, Merced College, and Queensborough 
Community College transcript data. 

Credits Earned Outside the Targeted Subject Area 

Figure 3.2 displays each college’s estimated impact and 95 percent confidence interval 
on the outcome progress outside the targeted subject during the program semester. As the 
graphic shows, there is little observed variation in impacts on this outcome across colleges — 
the amount of observed variation is no greater than what is expected by chance. 22 This suggests 
that across the six studied colleges, learning communities had a fairly homogenous average 
effect on students’ progress outside the targeted subject. In other words, the pooled impact 

22 A joint F-test yields a p- value equal to .24. 
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estimate basically tells the full story — there is not strong evidence that these programs help (or 
harm) students with respect to credit accumulation outside the targeted subject. 

Total Credits Earned 

Figure 3.3 displays each college’s estimated impact and 95 percent confidence interval 
on the outcome total credits earned during the program semester. This outcome represents the 
sum of the credits earned in and outside the targeted subject and may be the best proxy of 
students’ overall progress toward a degree. There is not meaningful evidence that the effect of 
learning communities varied across colleges on this outcome. That is, the average effect of 
learning communities was fairly homogenous across the six studied colleges. During the 
program semester, students who were offered the opportunity to participate in learning commu- 
nities earned an average of around 0.53 credits more than similar students without that oppor- 
tunity. While in Figure 3.3 there is some appearance that the impacts vary by college, the 
amount of observed variation in impact estimates is not greater than what would be expected by 
chance if the true impacts were the same across all colleges. 23 

Given that there is not meaningful evidence that the programs’ effects vary across col- 
leges, the pooled effect provides a useful estimate of the average effect of learning communities 
at these six colleges — masking little difference in average impacts among the colleges. It is 
important to be aware that the programs’ effects still may vary within colleges based on stu- 
dents’ characteristics or fidelity of implementation; however, this analysis suggests that among 
these six colleges the programs’ average effects on total credits earned were fairly homogenous. 

Notably, five of the six colleges have control group members whose average total cred- 
its earned range between 5.03 and 6.49 credits; in contrast, at Kingsborough control group 
members earned an average of 9.76 credits. Kingsborough’s students, regardless of learning 
communities, simply earned more credits than the students at the other colleges. As a result, if 
the impacts on total credits earned are considered relative to control group levels, then the 
impacts at Merced (12 percent) and Queensborough (11 percent) seem closer to those at 
Kingsborough (14 percent). This point is raised simply to call the readers’ attention to this issue, 
which can easily be lost in Figure 3.3. 

Impact Variation by Baseline Characteristics (Subgroup Analysis) 

The programs’ estimated impacts on the key outcomes are presented by subgroup in 
Appendix Tables B.8 through B.10. Like the college-by-college findings presented in the 
previous section, a certain amount of variation in impacts between subgroups is expected due to 


23 A joint F-test yields a p- value equal to .18. 
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The Learning Communities Demonstration 
Figure 3.3 

Impact of the Learning Communities Program on Total Credits 
Earned at the End of the Program Semester, by College 

Final Report of the Learning Communities Demonstration 
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SOURCE: MDRC calculations from the Community College of Baltimore County, Hillsborough Community 
College. Houston Community College, Kingsborough Community College, Merced College, and Queensborough 
Community College transcript data. 


chance alone (this is particularly true for subgroups with a small sample size). If the amount of 
observed variation in impacts is significant, then it is helpful to understand whether the learning 
communities program was particularly helpful (or harmful) for specific subgroups of individu- 
als. Such information could be useful for targeting this type of program toward those individu- 
als who are most likely to benefit from it. Described first in this section are a few subgroups that 
were specified before analyzing the data for this report. These are followed by additional 
subgroups that are exploratory and intended for hypothesis generation. 
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Race and Gender 


The first subgroup explored breaks students into eight race-by-gender combinations. 
This grouping is of interest since some subgroups of individuals have been historically disad- 
vantaged, and overall academic success rates tend to vary based on race and gender. Examining 
outcome levels (ignoring impacts) within the program group or control group shows that this 
sample conforms to some common trends in community colleges. Across all three outcomes, 
women tend to outperform men, and whites and “others” (comprised mostly of Asians) tend to 
outperform blacks and Hispanics. While outcome levels vary across these different subgroups, 
there is not strong evidence that the programs’ effects vary across these subgroups. 24 In other 
words, no race-by-gender subgroup clearly experienced more (or less) than the average impacts 
of these programs. 

Recent High School Graduate 

The second subgroup explored in Appendix Tables B.8 through B.10 is those students 
who graduated from high school or received a General Educational Development certificate 
(GED) within one year of entering the study, and those who did so one year or more before 
entering the study. Delayed enrollment between high school graduation and postsecondary 
enrollment has been identified by researchers as a “risk factor.” 25 Examining outcome levels 
(ignoring impacts) within the program group or control group demonstrates that indeed, those 
students who delayed enrollment tended to accumulate fewer credits than those who graduated 
or earned a GED within one year of the study. While outcome levels vary across these two 
groups, impacts are strikingly similar for recent high school graduates and those who delayed 
enrollment. 

Additional Subgroups 

In addition to the above main subgroups, four other binary subgroups were explored 
based on students’ answers to questions asked just before they were randomly assigned: 

• First in family to attend college 

• Usually speaks a language other than English at home 

• Single parent 


2 'Since some subgroups are fairly small (for example, “other males”) readers should take caution when 
considering the magnitude (and statistical significance) of the estimated impact for the smaller subgroups. 
However, given the total sample size, this study is well positioned to detect variation in impacts between these 
subgroups, even though some are relatively small. 

" Hoachlander, Sikora, Horn, and Carroll (2003); Horn, Berger, and Carroll (2005). 
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• Parents pay for more than half of expenses 

These additional subgroups were identified after sharing an earlier draft of this report 
with national experts in postsecondary education. Discussion with these experts led to the 
hypotheses that learning communities might be more or less effective for traditional students 
(leading to the exploration of the subgroups based on single-parent status and parents paying for 
more than half of expenses), students who were the first in their family to attend college, and 
students who usually speak a language other than English at home. Although the authors do not 
have a strong theory regarding why learning communities might work better for some of these 
groups, it seems plausible that certain groups of people might benefit more than others from the 
opportunity to enroll in classes with a cohort of students, or from receiving additional student 
supports. Perhaps this is especially true for subgroups that are the least likely to feel connected 
to their institution, those that might benefit the most from a peer support network, or those that 
are unsure how to navigate life in the community college setting. 

As shown in Appendix Tables B.8 through B.10, there is not meaningful evidence that 
the programs’ estimated impacts vary depending on whether students are the first in their family 
to attend college, whether they usually speak a language other than English at home, or whether 
they are single parents. 

In contrast, there is some evidence that the program may have been more effective for 
students who are financially dependent on their parents (that is, students who reported that their 
parents pay for more than half of their expenses). Analyses presented in Appendix Tables B.8 
through B.10 show that during the program semester, learning communities’ average impact on 
students who were financially dependent on their parents is statistically significantly greater 
than the program’s impact on students who were not financially dependent. This was true for 
credits earned in the targeted subject as well as total credits. Cumulatively after three semesters 
(not shown in tables), the differential impacts remain, with the program’s effects continuing to 
be larger on all three main outcomes for students who were financially dependent. 

Notably, at Kingsborough, where learning communities’ estimated impacts are the larg- 
est, over 70 percent of study participants reported being financially dependent on their parents at 
the time the study began. In contrast, at the other colleges, between 35 and 53 percent of study 
participants reported that they were financially dependent on their parents. 26 This might lead one 
to wonder whether financially dependent students appear to benefit more from learning com- 
munities because this type of program is more effective for them, or whether they attend 
Kingsborough at a higher rate and Kingsborough’s estimated effects are the largest. To try to 
answer this question, a test was conducted to determine whether the programs’ impacts tend to 


26 These percentages are among those who answered the question. 
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be larger for financially dependent students within the six study colleges, controlling for each 
college’s overall average impact. This test finds that the program’s effects are larger for finan- 
cially dependent students, even after accounting for each college’s overall average impact. 
These analyses suggest that the differential impacts by financial dependence are not simply an 
artifact of Kingsborough’s enrolling more financially dependent students. In contrast, it offers 
one hypothesis for why Kingsborough’s estimated impacts are larger than those of the other 
colleges — Kingsborough may serve more of the type of student (financially dependent) who 
benefits most from learning communities. Since many subgroups were explored, these analyses 
are intended for hypothesis generation, rather than as a definitive analysis. 27 

Impact Variation by Cohort 

Appendix Figures B.l through B.3 display the estimated impacts on the three key out- 
comes of interest by each sequential cohort within each college. For example, Hillsborough’s 
first cohort entered the study in fall 2007, the second cohort entered the study in spring 2008, 
and the third cohort entered the study in fall 2008. Owing to some of the challenges associated 
with participating in a random assignment evaluation and scaling up a program, there might be 
variation in the programs’ effectiveness across time or cohorts within the colleges. Specifically, 
program impacts might increase as the college “matures” in its implementation of learning 
communities. In fact, one of the main findings from the implementation research conducted at 
all six sites was that the learning communities tended to be more advanced as the colleges 
gained more experience in implementing learning communities at scale while taking part in a 
randomized experiment. 28 

Appendix Figures B.l through B.3 examine the hypothesis that impacts vary by cohort 
by plotting impact estimates within college across cohorts. If, for example, the programs were 
not “evaluation-ready” at the beginning of the study, but improved over time, one would expect 
to see impacts growing more positive for later cohorts in the study. This would be evidenced by 
generally upward-sloping impact estimates within each college. Notably, due to chance differ- 
ences, even if there was a true relationship between cohort and program impact, perfect trends 
should not be expected. However, if impacts improve in later cohorts, a general upward trend 
could be observed within the colleges across the cohorts. 

Across the three key outcomes of interest, there is not strong evidence supporting the 
hypothesis that as the programs matured, estimated impacts improved. This provides some 
indication that the fairly modest estimated impacts are likely not a result of programs being 
studied in their infancy. 


27 See Schochet (2008) for more information on multiple testing and “exploratory” analyses. 
28 Visher, Schneider, Wathington, and Collado (2010). 
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Program Participation 

Another important consideration when examining program effects in a randomized experiment 
is the level of program participation. In many social experiments, individuals do not comply 
perfectly with their program assignment, and the learning communities studies were no excep- 
tion. Across the six participating colleges, 71 percent of program group members were still 
enrolled in a learning community at the add/drop deadline. Stated differently, 29 percent of 
study participants assigned to the program group were not enrolled in a learning community at 
the add/drop deadline (in experimental nomenclature, these students are sometimes referred to 
as “no-shows”). The main analyses presented in this report are intention-to-treat, meaning they 
compare outcomes for all program group members with outcomes for all control group mem- 
bers and represent the impact of the opportunity to participate in learning communities, not 
necessarily the impact of experiencing the program. 29 Many readers may be interested in the 
effect of the program only on those who participate, or the treatment-on-treated. While methods 
are available to estimate treatment-on-treated, 30 the assumptions required for such analyses may 
not be met in this study. 31 

In order to attempt to come closer to estimating the effect of learning communities on 
those students that experienced learning communities, nonexperimental analyses were conduct- 
ed on only those students (program group and control group) who enrolled in at least one class 
by the add/drop deadline of the program semester (see Appendix Table B.ll). The percentage 
of program group members enrolled by the add/drop deadline is similar to the percentage of 
control group members enrolled at the add/drop deadline, so it is possible that these groups were 
similar at the outset of the study. 32 This subset of students is of interest because they had a 
significantly higher rate of program take-up, or participation in learning communities (84 
percent, compared with 71 percent for the full sample). As a result, analyses using this subset of 
students may provide a closer estimate of the impact of the learning communities program on 


2 9 There were very few control group members who managed to enroll in learning communities, hence 
there is no discussion of “crossovers” in this report. 

30 See, for example. Bloom ( 1984) and Angrist, Imbens, and Rubin ( 1996). 

3 1 Specifically, such analyses require the assumption that the intervention has no impact on those randomly 
assigned to the program group who are considered not to have experienced the intervention (that is, the “no 
shows”). In this case, this assumption is strong, since program participation is defined several weeks into the 
semester, at a point when the program may already have affected a student’s outcomes, for better or for worse, 
hi addition, participation is defined as “co-enrolling in a learning community link,” but some colleges offered 
additional intervention services, which a student might benefit from even if not co-enrolled in the learning 
community links. 

2 In other words, as long as students’ assignment did not affect their likelihood of remaining enrolled at 
the add/drop deadline, these analyses can still be viewed as experimental. However, if students’ assignment 
changed the mix of types of students enrolled at the add/drop deadline, these analyses could provide a biased 
estimate of the treatment on enrollees. 
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those students who actually participated in them. These “sensitivity” analyses confirm the 
pooled findings and result in almost no change in the magnitude of the pooled impact estimates. 
This provides some evidence that the overall analyses presented in this report might not be too 
different from the effect of learning communities on those who participated in them. 

These sensitivity analyses do offer some evidence that the programs’ effects varied 
across colleges on total credits earned during the program semester; however, by the end of the 
third semester there is still not meaningful evidence of variation in impacts among colleges on 
total credits earned. 

Conclusion 

This chapter presents findings on the overall pooled effectiveness of six one-semester learning 
communities programs targeting developmental education students, operated at six community 
colleges throughout the United States. Aggregated across these different contexts, programs, 
and students, learning communities caused students to earn one half-credit more than similar 
students who were offered their college’s usual services. This impact on total credit accumula- 
tion is a result of students earning more credits in a specific, targeted developmental subject (the 
subject varied by college). There is evidence that the programs’ average effects on credits 
earned in the targeted subject varied across the six colleges; however, the average effect of 
learning communities on overall academic progress was fairly homogenous among the colleges. 

These six community colleges and programs were not selected randomly from a popu- 
lation of community colleges implementing learning communities programs. It is possible that 
the effects of learning communities at other colleges are different from those described and 
studied in this report. In other words, since the results presented in this report are from six 
community colleges, claims regarding the effectiveness of learning communities at other 
colleges are speculative. 

That said, in the absence of other rigorous estimates of the effects of other learning 
communities programs, the research presented here may provide the best evidence to date on 
the average effectiveness of learning communities programs more broadly, even though this 
evidence has a lot of uncertainty. The fact that these six programs, serving diverse students, 
running variations of learning communities, have fairly homogenous effects on average total 
credit accumulation, should engender more confidence in the generalizability of these findings. 

In sum, the evidence presented in this report finds that learning communities have a 
small but positive effect on students’ overall academic progress. The next chapter of this report 
synthesizes the implementation research that was conducted in each college to discern patterns 
that might explain these results. This is followed by a chapter describing the costs associated 
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with operating this type of program, which provides invaluable context for considering the 
impacts of this type of program. 
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Chapter 4 

Understanding the Impacts in Light of the 
Implementation Research 


The analysis in the last chapter pooled transcript data across the six sites and found that, on 
average, one-semester learning communities for developmental education students produce an 
estimated impact of about half a credit on total credits earned within three semesters after 
students enrolled in the study. The impact occurred mostly in the first semester and mostly from 
courses in the targeted subject (English or math). Although the evidence is somewhat mixed, 
the analysis further suggests that results for the program at Kingsborough Community College 
stood out from the others, with a larger impact on credits earned in the targeted subject area. 
Additional findings included evidence that students who were enrolled in the learning commu- 
nities after the programs matured generally fared no better than students enrolled earlier during 
the scale-up period. Finally, the analysis showed that learning communities do not lead to 
higher rates of reenrollment during the three semesters of follow-up. 

Colleges operate learning communities with the expectation that students will have bet- 
ter outcomes in part because they will experience stronger relationships with their fellow 
students and their instructors and engage more deeply in the content of courses when they are 
connected with others. Low or no impacts from interventions of any kind can come about for a 
variety of reasons. 1 Based on a synthesis of the implementation research conducted across all 
six sites, this chapter explores two possible explanations for the modest impacts described in 
this report: 

1. The learning communities programs may not have been implemented with fidelity 
to the model, or there may have been a large degree of variation in implementation 
either within or across programs — or both — leading to a dilution of the effects of 
learning communities. 

2. The programs may have been implemented with reasonable fidelity, but the control 
groups could have received services or experiences that resembled those received 
by the program group, thereby reducing the impact (defined as the difference be- 
tween the outcomes of the two groups). 

A third possible explanation was explored in Chapter 3: The programs may have been 
implemented with reasonable fidelity, and the treatment contrast may have been sufficient, but 


'Dane and Schneider ( 1998); Lipsey and Corday (2000). 
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students in the program group may have failed to enroll in the program at a rate that allowed a 
significant number of the students to experience learning communities and thereby get the 
appropriate “dosage” of the treatment. An analysis of the outcomes of a subset of students in the 
study — those who were enrolled in at least one course by the add/drop date — suggests that 
despite somewhat low participation rates by the program group in some sites, it does not appear 
that this had a large effect on the pooled impact findings. 

The key findings for this chapter are: 

• Overall, the learning community programs in the six colleges were imple- 
mented with reasonable fidelity to the models, although in most colleges 
program implementation was uneven in the first semester or two as the col- 
leges worked to scale up and stabilize the programs. 

• Curricular integration, a key component of the model, was not routinely ob- 
served across the learning communities in the programs. There was consider- 
able variation within each college in the extent to which courses in the learn- 
ing communities were integrated through aligned syllabi, joint assignments, 
or other integrative strategies. 

• Generally, the control group seemed to have had a different experience from 
the program group. The control group was less likely to take courses with the 
same students and had less access to certain types of student services. 

• The Kingsborough program featured several enhancements that may account 
for the somewhat larger impacts on credits earned in the targeted subject area 
compared with the other programs. 

The remainder of this chapter is organized into four sections. First, a synthesis of the 
findings from the implementation research across the six sites is presented. Second, the extent to 
which implementation varied across and within the six sites and over time is discussed. Third, 
the question of how the experiences of the program group differed from that of the control 
group is examined. Finally, several explanations are offered to help shed light on why the 
Kingsborough program produced larger impacts on credits earned in the targeted subject area. 

Synthesis of Findings on the Implementation of the Programs 

Typical learning communities for students in developmental education in community colleges 
tend to share some common features, although they vary in how much emphasis is placed on 
particular components. They generally last one semester and enroll a group of first-year students 
together in two or three linked courses, at least one of which is a developmental English or math 
course. Some are loosely organized under an overarching theme. They are taught by faculty or 
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adjunct instructors who are generally expected to communicate with each other at least a few 
times each semester about both the course content and the progress of their shared students. 
Learning communities sometimes include curricula that are linked in some way, for example by 
an overarching theme, through joint assignments, and some alignment of the syllabi. Instructors 
use pedagogical practices to encourage active and collaborative learning, taking advantage of 
the stronger relationships among students in the class that occur through this “cohorting” 
experience. Students typically are offered some form of extra academic support, such as 
tutoring or advising, over and above what is usually offered. Some programs link a student 
success course with a developmental course to provide this support as part of the learning 
community itself. 

Accordingly, the study team intentionally recruited colleges operating a range of learn- 
ing community models that represented the continuum of developmental education learning 
community programs as typically offered at community colleges. The resulting six programs 
examined in this report — at the Community College of Baltimore County (CCBC), Hills- 
borough Community College, Houston Community College, Kingsborough Community 
College, Merced College, and Queensborough Community College — represent a broad range 
of program models that together reflect this continuum of “real world” learning communities at 
community colleges. While they shared certain features, such as linking a developmental course 
with other courses and incorporating some form of extra support for students into the model, 
they varied in other ways according to the goals and intentions of the colleges. 2 Table 4.1 
presents the key findings from the implementation research conducted at each of the six sites. 
(Readers are encouraged to refer to the site-specific publications listed at the end of this report 
for more detailed descriptions of the data collected at each site that led to these conclusions.) 
Several patterns are evident: 

• All six sites were able to successfully link a targeted developmental course 
with another developmental or college-level course (and in Kingsborough’ s 
case, two other courses) and enroll a group of students in these linked cours- 
es. All told, 174 learning communities were operated, in which nearly 4,000 
students were randomly assigned to the program group and invited to enroll 
in a learning community. 

• Faculty collaboration generally occurred to the extent expected by the pro- 
gram leaders in most of the learning communities, although more collabora- 


2 While the team attempted to find at least one program that was close to the “ideal” as described in theo- 
retical work by experts, this proved difficult. Two colleges among those discussed in this report, Kingsborough 
Community College and Merced College, came the closest to the ideal model. 
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The Learning Communities Demonstration 
Table 4.1 


Summary of Selected Findings from Implementation Research 
on the Learning Community Programs 

Final Report for the Learning Communities Demonstration 

College Selected Implementation Findings 

The Community College of Baltimore • The program featured a unique student support component, the 
County (CCBC) Master Learner session, which was often taught by the same instructor 

(Baltimore, MD) who taught the developmental course. No credits were associated with 

this session. The content varied considerably by instructor, but most 
used it to assist students in developing reading and writing skills as 
well as mastering the course content necessary for success in the 
college-level course in the link. 

• The number of learning communities had to be increased to meet the 
requirements of the study design. This pressure meant that for much of 
the time the program was operated it was in some flux. 

• After the first semester of the program, administrators provided 
increased structure and professional development for faculty, but there 
continued to be significant variation in implementation among the 
learning communities in the program, particularly in curricular 
integration and faculty collaboration. 


• The program featured a link with a student success course. 

• Faculty and students reported a sense of community and increased 
social interaction between students. 

• Integration of curriculum in the two courses in the link was not 
initially emphasized by program leaders but by the last semester of 
study intake, more instructors were attempting to align and integrate 
their courses. 

Houston Community College • The program featured a link with a student success course and field 

(Houston, TX) trips. 

• Many students reported a strong sense of community and belonging in 
learning communities. 

• Integration of curriculum in the two courses in the link was not 
initially emphasized by program leaders but as the program evolved, 
most instructors worked with each other to create joint assignments. 

(continued) 


Hillsborough Community College 
(Tampa, FL) 
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Table 4.1 (continued) 


Kingsborough Community College • The program featured a three-course link, including a “freshman 

(Brooklyn, NY ) orientation course,” whose instructors functioned as “case managers,” 

and a college-level course; there was enhanced advising; tutors were 
assigned to each learning community; and book vouchers were 
provided. 

• Students in learning communities could earn a relatively large number 
of credits through the three-course link and received a book voucher as 
an incentive to enroll in the intersessions, giving them access to even 
more credits in their first year. 

• College leaders were consistently supportive of the program. 

• The learning communities varied in their content, class sizes, and in 
the degree to which faculty worked together and integrated their 
courses. 

Merced College * The program featured a relatively high level of faculty team 

(Merced, CA) collaboration, as instructors were generally active in creating linked 

syllabi, visiting each others’ classrooms, and discussing individual 
students' progress. 

• Learning community instructors and students considered the 
enrollment of students in cohorts — and the stronger relationships that 
resulted — to be the most salient component of Merced's learning 
communities. Students participating in learning community courses 
reported that they experienced stronger relationships with their cohort 
peers and instructors. 

• Many of the learning communities had overarching themes and high 
levels of cross-course content integration. However, integration varied 
among the learning communities. Those faculty teams who had taught 
together for a semester or more collaborated more than those who were 
new to learning communities. 

Queensborough Community College • The program featured a link with a developmental or college-level 

(Queens, NY) course. 

• While the learning communities varied in how much instructors 
attempted to integrate their courses, by the second semester of study 
intake, more instructors collaborated on joint assignments and 
overarching themes. 

• The learning community coordinator reported staying in close touch 
with many of the learning community students to encourage regular 
attendance and to offer advisement. 

SOURCES: Scrivener et al. (2008); Weiss, Visher, and Wathington (2010); Weissman et al. (2011); Weissman et al. 
( 2012 ). 

tion occurred later in the demonstration, as those expectations changed over 
time. 

• Efforts to integrate the course content in the links by aligning syllabi and as- 
signments across the courses in the learning communities varied, but there is 
some evidence that this component became stronger over time. 
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Extra support was generally offered as intended, although support other than 
that provided in linked student success courses was fairly minimal. 


Variation in the Implementation of Learning Communities 

Even when interventions as a whole are implemented with a reasonable degree of fidelity, 
variation may occur in how practices associated with particular components of an intervention 
are delivered to different participants. 3 The primary challenge to fidelity of implementation in 
the six sites of this study was a degree of inconsistency with which one key component of the 
intervention — faculty collaboration and curricular integration — was implemented. There was 
considerable variation in how tightly the courses were linked in the learning communities in all 
six programs. 

Tight integration of the content of linked courses has been described as the most diffi- 
cult component of this model to implement. 4 As earlier reports from both the Learning Com- 
munities and Opening Doors demonstrations documented, variation in practices associated with 
faculty collaboration in curricular integration were indeed considerable, particularly in the first 
one or two semesters of program implementation. In fact, at least as much variation occurred 
within colleges as across colleges through at least the first year of implementation. 5 Results 
from a survey conducted with instructors in the learning communities underscore the difficulties 
some faculty had in implementing this component. 6 While over half of the learning community 
instructors who responded to the survey reported that they believed it was “very important” to 
collaborate with other instructors on syllabi or assignments, only 18 percent reported that they 
actually did so more than five times per semester (see Table 4.2). 

Based on interviews conducted with faculty and college leaders, there is some indica- 
tion that attempts among learning community instructors to collaborate with each other to 
integrate the curricula across the linked courses increased after the first semester or two of 
program operation, likely as a result of professional development opportunities, clearer expecta- 
tions from the learning communities coordinator, and increased faculty engagement. 7 For 
example, Kingsborough experienced challenges, especially in the first semesters of the 


3 Lipsey and Corday (2000). 

Wisher, Schneider, Wathington, and Collado (2010). 

Wisher, Schneider, Wathington, and Collado (2010). 

6 Because random assignment of teachers to either learning communities or stand-alone classes was not 
possible, it is impossible to know if any of the differences discussed were owing to differences in the types of 
teachers who volunteered to teach in a learning community or to changes in practices and beliefs that occurred 
as a result of teaching in learning communities. More information about the faculty survey is provided in 
Appendix C. 

Wisher, Schneider, Wathington, and Collado (2010). 
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The Learning Communities Demonstration 
Table 4.2 

Selected Items from the Instructor Survey 
Final Report of the Learning Communities Demonstration 



LC 

Non-LC 


Instructor 

Instructor 

Status (%) 



Full time (tenure or tenure-track) 

55.2 

42.9 

Full time (temporary or adjunct) 

12.4 

10.9 

Part time (temporary or adjunct) 

32.4 

46.3 

Beliefs about teaching: How important are the following activities for 
being an effective teacher? (% Very important) a 



Participating in professional development for teaching 

66.7 

62.2 

Collaborating with other instructors on syllabi or assignments for your course 

54.3 

45.6 

Communicating with other instructors about shared students 

58.3 

38.9 

Helping students make connections between your course and other courses 

79.1 

65.8 

Engagement and collaboration: How frequently do you do the following? 
(% More than 5 times per term ) b 



Discuss teaching practices with other instructors at your institution 

44.2 

38.7 

Collaborate with instructors on syllabi or assignments for your course 

18.3 

10.2 

Communicate with other instructors about shared students 

35.2 

16.8 

Discuss course content with other instructors at your institution 

34.0 

23.0 

Student support services: How frequently do you do the following? (% More 
than 5 times per term) 



Have student services staff attend your class 

5.7 

5.4 

Refer students to services or resources on campus 

47.1 

28.2 

Academic support: How frequently are the following services provided 
during scheduled class periods? (% More than 5 times per term) 



Supplemental instruction 

19.2 

15.0 

Tutoring 

18.3 

20.6 

Student study groups/labs 

19.8 

18.8 

Writing/reading support 

19.6 

17.0 

Professional development: How frequently have you participated in professional 
development activities in 2007-08 (% More than 5 times per term) 

26.0 

16.3 

Sample size (total = 255) 

105 

150 


(continued) 
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Table 4.2 (continued) 


SOURCE: MDRC calculations from a survey of instructors at the Community College of Baltimore County, 
Hillsborough Community College, Houston Community College, Merced College, and Queensborough 
Community College. 

NOTES: Rounding may cause slight discrepancies in sums and differences. The results are reported here for 
descriptive purposes only. The response rate for learning community instructors was 82 percent and the response 
rate for non-learning-community instructors was 52 percent. Both the overall response rate and the fairly large 
difference in response rate between instructors who taught in learning communities and instructors who did not 
teach in learning communities reduces confidence in the inferences that can be drawn regarding these two 
instructor groups, and how they might differ. Additional information on this survey is provided in Appendix C. 

“Response items comprise a 3-point scale: “Very important,” “Somewhat important,” and “Not at all 
important.” 

b Response items comprise a 4-point scale: “More than 5 times per term,” “2-5 times per term," “Once per 
term,” and “Not at all.” 

program. The compressed time frame for beginning the study forced the administrative leaders 
at Kingsborough to rush in recruiting and matching up faculty in teams to teach the learning 
communities. As a result, curricular integration across the program was spotty at best. But with 
time these early problems were ironed out as more faculty began to step forward and pair with 
other faculty with whom they had preexisting professional relationships or shared interests. This 
change, combined with improved opportunities for faculty development, led to a gradual 
increase in the degree of curricular integration that took place within the learning communities . 8 

In sum, within the first or second semesters, most colleges were able to fully implement 
the key components of their programs, with the notable exception of the integration component. 
While examples of curricular integration became more prevalent later in the project, this 
element rarely reached a level in most learning communities where it became a defining feature 
of the teaching or learning experience, according to interviews with both instructors and 
students. Because the impact estimates in this report are averaged across all learning communi- 
ties in each college as well as across the colleges, the modest impacts may in part due to the 
rarity of tightly integrated programs. To know with any confidence that stronger curricular 
integration would have made a difference would require comparing the outcomes of students in 
learning communities that had strong exposure to integration with those that had weak exposure 
to integration. Such an analysis was beyond the scope of this study. 

Challenges in Scaling Up Learning Community Programs 

New or revised programs may take years to get up and running in the intended manner, if, 
indeed, that is ever accomplished . 9 To conduct a fair test of interventions, it is important that 


8 Scrivener et al. (2008). 

9 Lipsey and Cordray (2000). 
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programs are as mature and stable as possible during the evaluation. Newer programs may 
implement some programmatic elements but may neglect or poorly attend to others. 

In the demonstration, most of the colleges were asked to scale up their learning com- 
munity programs in order to generate the number of students required for the study. In some 
cases, colleges had to “grow their programs” from the two or three learning communities they 
had previously run to as many as five or six. While this was undoubtedly a challenge, running 
learning communities was not new to any of the colleges, and all six were interested in increas- 
ing the number they could offer. CCBC, Kingsborough, and Merced had conducted learning 
communities programs for several years before joining the demonstration; the other three had 
somewhat less experience. 10 

However, even the colleges that had considerable experience with learning communi- 
ties experienced a period of instability while the programs were being scaled up. Challenges 
included recruiting and training faculty, registration and block scheduling, student take-up and 
participation, and pairing the “right links.” 11 To test the possibility that students who participat- 
ed in learning communities in the early semesters of the program may have experienced weaker 
programs than those who participated in late semesters, a comparison of impacts across cohorts 
was reported in Chapter 3. The results suggest that later cohorts of students in the program 
group did not seem to benefit more than earlier cohorts (compared with the control groups). The 
pattern across the colleges suggests that if program maturation occurred over the course of the 
demonstration, it did not significantly change the average impact of the program. 

Differences in the Experiences of the Program and Control 
Groups 

Another possible explanation for the modest impacts of learning communities on students’ 
outcomes is that the control group at some or all of the colleges may have had experiences that 
were similar to those of the program group. A review of the findings from the implementation 
research conducted at each college suggests that on the whole, the control group students had 
experiences that were distinct from those of the program group. Students attending regular 
classes were less likely than program students to be exposed to learning environments where 
they formed relationships with other students in their classes, less likely to have access to 
certain forms of support, and less likely to have instructors who intentionally made connections 
with the content of other courses or collaborated with other instructors. 


10 Visher, Schneider, Wathington, and Collado (2010). 
n See Visher, Schneider, Wathington, and Collado (2010). 
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Linked Classes and Student Cohorts 

Learning community students were consistently grouped into linked classes with com- 
mon instructors: all or almost all of the students enrolled in one linked course were also enrolled 
in the other(s). Moving from class to class as a cohort was an experience that was substantively 
different from that of traditional stand-alone classes and tended to be the most salient feature of 
learning communities that students in all of the sites noticed and appreciated (see Table 1.2 for a 
description of the linked courses). 

Student Support Services 

All six programs offered some form of extra support to assist students that went beyond 
the “business as usual” services offered to students enrolled in traditional developmental 
programs. Four programs (CCBC, Hillsborough, Houston, and Kingsborough) embedded this 
support in the learning community itself by including a course or seminar in the link designed to 
provide academic support, study skills, and other information to help students better navigate 
college. One program encouraged use of the tutoring center (Houston), and students in two 
programs had access to extra advising and counseling (Kingsborough and Queensborough). 
Support services offered at Kingsborough were particularly strong, as is discussed in more 
detail below. 

Additional evidence that the program group had access to more or different types of 
support than the control group is also available from the responses of instructors to the survey 
(see Table 4.2). Learning community instructors were more likely than non-learning- 
community instructors to report that they referred students to services or resources: Forty-seven 
percent of learning community instructors said that they referred students to services at least 
five times a semester, compared with 28 percent of non-learning-community instructors. More 
learning community instructors (58 percent) indicated that they believed it was very important 
to communicate with other instructors about students’ progress than non-learning-community 
instructors (39 percent). On the other hand, incorporating extra academic support in the class- 
room, such as tutoring and supplemental instruction, was relatively rare in both settings. 

Faculty Collaboration on Integrating the Curriculum 

Students in traditional classes were somewhat less likely to have instructors who often 
collaborated with other instructors on course content or the progress of individual students, 
according to comments made by both students and instructors in focus groups and the responses 
of instructors to the survey questions. For example, 79 percent of learning community instruc- 
tors reported believing that it was very important for students to make connections between 
their course and other courses, compared with 66 percent of non-learning-community instruc- 
tors. Fifty-four percent of learning community instructors believed that it was very important to 
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collaborate with other instructors on course syllabi and assignments, compared with 46 percent 
of non-leaming-community instructors. More learning community instructors reported that they 
frequently engaged in such collaboration, compared with those teaching in traditional classes. 
However, doing so frequently was rare even in the learning community group, with only 18 
percent reporting collaborating more than five times a semester. 

The Kingsborough Program 

The findings reported in Chapter 3 suggest that the learning communities program at Kings- 
borough had a larger estimated impact on credit accumulation in the targeted subject area than 
the other learning communities programs. Kingsborough’ s impact on this outcome was sus- 
tained for at least three semesters. There was not a statistically significant difference in impacts 
between the six programs on total credits earned or credits earned outside the targeted subject. 
Several features of the setting and program at Kingsborough may help explain the difference in 
impacts observed in credits earned in the targeted sequence. 

First, the program model included some significant enhancements compared with the 
other five models, particularly in the area of student support services. Compared with students 
who were given the opportunity to enroll in the programs at the other five colleges, Kings- 
borough students were offered an array of services to help them succeed in their courses, persist 
in their studies, and make good choices about what courses to take. The support began with 
vouchers to buy books and continued with a second book voucher if they enrolled in the 
intersession immediately following the learning community semester. Students were actively 
encouraged to enroll in the intersession. Each learning community was assigned a tutor who 
attended class. One class in each link (the freshman orientation class, which was comparable to 
the student success courses in several of the other programs) was taught by a counselor who 
served as a case manager during the program. Finally, a special office staffed with advisers was 
set up to support learning community students, and although students were encouraged to use 
regular advising services after the learning community was over, the office maintained an 
“open-door” policy for former learning community students. No other colleges in the Learning 
Communities Demonstration came close to this level of student support. 

Second, Kingsborough’ s learning communities were associated with a higher number 
of credits than learning communities at the other colleges. The average number of credits 
students could earn in a learning community ranged from 5.0 to 7.3 at the five other sites. At 
Kingsborough, the learning communities were associated with, on average, 10.1 credits. 

Third, the Opening Doors program enjoyed unusually strong support from college ad- 
ministrators, from the president down to department chairs, surpassing that of any of the other 
colleges. Faculty teaching in learning communities “got the message” that their work was 
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important and valued, which may have attracted the most effective teachers to the program or 
motivated those who taught in them to perform at their best. 


Summary 

While the implementation of learning communities across the six colleges was not without 
challenges, taken as a whole, the programs were implemented as designed and intended by the 
colleges. This synthesis of implementation research reveals several common patterns. First, the 
learning communities at the colleges strengthened over time but, as demonstrated in Chapter 3, 
this did not seem to result in stronger impacts over time. Second, one component of the model, 
curricular integration, was rarely implemented to its full potential, even in those colleges that 
emphasized this component in the model. Third, the experience of the control group was 
distinct from that of the program group in several critical ways. Students given the opportunity 
to enroll in a learning community were much more likely to take courses with the same group of 
students. They were also more likely to have faculty who collaborated with each other and tried 
to help students see connections between their courses. 

Because curricular integration was not a well-developed or consistent feature of the ob- 
served learning communities, and the study was not designed to measure how variation in 
implementing this component was associated with student outcomes, it is difficult to say if or 
how the impacts might have changed if this component had been fully in place in most or all of 
the learning communities in the study. Nonetheless, implementation research findings suggest 
that the lack of large impacts described in the report cannot be easily “explained away” by poor 
implementation or a weak treatment contrast. 
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Chapter 5 

The Cost-Effectiveness of Learning Communities 


The analysis of impacts of learning communities described in Chapter 3 finds that one-semester 
learning communities for students in developmental education yield an average impact of about 
half a credit over three semesters. When considering the role learning communities can or 
should play in improving the outcomes of academically underprepared students, both the 
average expected impact and the cost of producing that impact need to be considered together. 
This chapter offers information about the average costs of educating a student in learning 
communities, compared with the average costs of educating the same student in regular classes 
with regular services. The chapter also includes a discussion of whether the additional costs of 
learning communities are justified given the size of the expected impact of the intervention on 
credits earned. The key findings in this chapter are: 

• The direct cost of operating a typical, one-semester learning community is 
around $570 per program group member during the semester in which it is 
operated. The most expensive component of learning communities is addi- 
tional instructor time, which is associated with almost $250 for each student 
(program group member) in the learning community. 

• When the additional costs of educating students are considered, over the 
three semesters of follow-up, the colleges invested slightly over $700 more 
per program group member than they did per control group member. 

• When program costs are compared with impacts of learning communities on 
credit accumulation, the intervention is similar but slightly less effective dol- 
lar for dollar than regular college services. Specifically, the average cost per 
credit earned is approximately $572 for learning community students and 
$544 for students receiving regular services — a difference of $29. 

Methodology 

This chapter estimates the cost-effectiveness of the Opening Doors program at Kingsborough 
and the five developmental education learning community programs in the Learning Communi- 
ties Demonstration. These costs are estimated using financial information from each college, 
and since the resources (such as tuition paid by students, government subsidies, and private 
donations) are generally not generated directly by the college but rather are funneled through 
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the college, this approach provides a good estimate of the financial investment of society at 
large in these programs. 1 All dollars are adjusted for inflation and displayed in 201 1 dollars. The 
analysis excludes costs that are associated with start-up activities and research. For purposes of 
presentation, all costs are classified as either direct costs, base costs, or indirect costs (see Box 
5.1 for definitions of terms used throughout this chapter). Additional detail and discussion 
related to the cost of operating learning communities is included in Appendix D. 

Direct Cost 

The direct cost of the program accounts for the resources required to operate learning communi- 
ties that are incurred during the semester in which the program is operated (the costs in subse- 
quent semesters are discussed below). Direct program costs are estimated using program 
participation and budget data from three of the six colleges included in this report (the Commu- 
nity College of Baltimore County [CCBC], Houston Community College, and Kingsborough 
Community College). 2 The direct costs are outlined in Table 5.1. 

Table 5.1 shows that the learning community programs required an investment of near- 
ly $570 per program group member during the program semester. 2 The largest cost, $246 per 
program group member, was associated with additional instructional costs. These costs included 
stipends for instructors who taught in a learning community to help compensate for the extra 
time needed for planning courses and collaborating with other faculty and the cost to the college 
of operating smaller classes (the role of class size is discussed in detail in Appendix D). The 
second-largest cost, $135 per program group member, was related to extra student support 


'Many benefit-cost analyses examine benefits and costs from multiple perspectives such as participants, 
taxpayers, the rest of society, and society as a whole. This cost-effectiveness analysis compares the investment 
of the college (which simultaneously serves as an estimate of society as a whole, because all of society is 
funneling its resources through the college) to the outcomes associated with that investment. 

2 A11 three sites use budget documents as the primary source to estimate direct program costs; however, the 
information available for each college is sometimes slightly different. For example, at Houston a survey was 
conducted to measure program group members’ use of the college’s support services, such as tutoring. Because 
it turned out that program group members used tutoring more than control group members, program group 
members were assigned a larger portion of these costs at Houston. Such a survey was not available at the other 
sites. An effort has been made to make the measurement of costs as comparable as possible across colleges but 
it is important to note that certain differences remain, making direct comparisons difficult. 

3 The Kingsborough direct cost estimate of $1,190 per program group member is slightly different from the 
cost estimate featured in a separate Kingsborough-specific report, where the direct cost estimate is $1,580 per 
program group member (Sommo, Mayer, Rudd, and Cullinan, 2012). This difference is partly a result of an 
attempt to make the results presented in this repoit as comparable as possible across sites. Specifically, the cost 
estimates at CCBC and Houston were based on the cost as observed during the second fall semester of program 
operation. The Kingsborough estimate in the Kingsborough-specific report is based on the average cost across 
four semesters of operation. An adjustment has been made to improve the comparability of the estimates 
between Kingsborough and the other sites. 
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Box 5.1 


Key Cost Terms 

Direct cost is the cost directly associated with providing program services during the 
period of program operation. 

Base cost is the cost of the college’s regular services in the absence of the program. Base 
cost is calculated by multiplying the cost per credit by the average number of credits 
attempted by the control group. The cost per credit is an estimate of the average amount of 
resources expended by the college to provide one credit’s worth of instructional activity; it 
is calculated by dividing the college’s annual operating budget by the number of credits 
attempted at the college during the year of interest. 

Indirect cost is the cost resulting from a behavioral change brought about by the program, 
such as additional credits attempted by program group members. Such costs can extend 
beyond the period of program operation. The indirect cost of the program is calculated by 
multiplying the cost per credit by the number of additional credits attempted by program 
group members. 

Program group cost is the total cost of educating program group members over three 
semesters of follow-up. Program group cost is the sum of direct cost, base cost, and 
indirect cost. This can be divided by the number of program group members to get the cost 
per program group member. 

Control group cost is the total cost of educating control group members over three 
semesters of follow-up. Control group cost is equivalent to base cost. It can be divided by 
the number of control group members to get the cost per control group member. 

Net cost is the difference in costs between program group members and control group 
members. Net cost is equal to the control group cost subtracted from the program group 
cost. 

Cost-effectiveness analysis is an assessment in which the net costs of an intervention are 
expressed as the cost per unit of a desired outcome. In this analysis, cost-effectiveness is 
presented for cost per credit earned. 

Cost per credit earned is the amount invested in the research group of interest per credit 
earned by that research group. For the program group, the cost per credit earned is calcu- 
lated by dividing the cost per program group member by the number of credits earned per 
program group member. 
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The Learning Communities Demonstration 
Table 5.1 


Direct Cost of Learning Community Programs: Average Costs per 
Program Group Member, by College (in 2011 Dollars) 


Final Report of the Learning Communities Demonstration 


Type of cost 

Average 

CCBC 

Houston 

Kingsborough 

Direct cost of program components ($) 





Additional instructional costs 

246 

225 

33 

569 

Program support and student services 

135 

24 

84 

338 

Administration 

96 

160 

62 

68 

Other 

89 

52 

31 

215 

Total ($) 

566 

461 

211 

1,190 


SOURCE: MDRC calculations from program-specific participation and budget data from the Community College 
of Baltimore County (CCBC), Houston Community College, and Kingsborough Community College. 

NOTES: Estimates have been adjusted for inflation using the Higher Education Price Index for public 2-year 
institutions. 

Rounding may cause slight discrepancies in sums and differences. 

Program costs are based on a steady state of operation that excludes research and start-up costs. 

Calculation is based on 1,955 program group members. 

Costs from the three colleges are weighted by number of program group members (650 at CCBC, 761 at 
Houston, and 544 at Kingsborough). Site-specific values are available in Appendix Table E.l. 

CCBC costs are based on the costs observed during the second fall semester of operation. This was the 
semester with the highest level of student participation in the learning communties (Weissman et al., 2012). 

Houston costs are based on the costs observed during the second fall semester of operation. This was the 
semester with the highest level of student participation in the learning communities (Weissman et al., 2011). 

Kingsborough costs are based on the costs observed during the second fall semester of operation. This was 
the semester with the highest level of student participation in the learning communities (Bloom and Sommo, 

2005). The primary cost estimate for the Kingsborough program in Sommo, Mayer, Rudd, and Cullinan (2012) 
looks at the average cost of the four semesters of operation. These data were not available for the other sites, so 
costs for the second fall semester at Kingsborough were used instead of the average cost over the four semesters to 
make the above estimates as comparable as possible. 


services for students in learning communities, such as a case manager or tutor assigned to work 
with students in some learning communities. Less than 20 percent of the program semester 
direct costs were related to administration ($96 per program group member); this covered 
stipends and/or salaries of various program administrators and coordinators and sometimes 
included costs of in-service training for instructors. 4 Lastly, $89 per program group member was 


4 Stipends paid during this period may be greater than what is needed in the long run ( this analysis does not 
adjust cost values based on changes the college may have implemented beyond the demonstration period). For 
example, some colleges used stipends or release time as an incentive to encourage new faculty to participate in 

(continued) 
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associated with a variety of other costs; one site gave students vouchers for books, while others 
purchased equipment or paid for the cost of transportation for field trips. The average direct cost 
of the program components are highlighted in the top row of Table 5.2. 

Base Cost 

In order to provide a basis for comparison for the costs of learning community programs, this 
next section estimates the cost of regular college services provided to all students, whether or 
not they participated in a learning community program. Accurately estimating this cost is 
challenging because of limitations in the data generally available, so this analysis uses the 
estimated cost of credits attempted as a proxy for base costs. This approach assumes that 
resource use corresponds to the number of credits attempted; in other words, a student who 
attempts more credits is generally associated with greater expenditures than a student who 
attempts fewer credits. 

To estimate the dollar value of credits attempted, the number of credits attempted is 
multiplied by an estimated cost per credit offered. This cost per credit is estimated by dividing 
the college’s annual operating budget by the number of credit hours attempted at the college 
during the year of interest. The average cost per credit yielded by this calculation is then used to 
estimate the cost of regular college services. 

This approach has limitations. One is the assumption that all credits attempted have the 
same cost, which is not necessarily the case. 5 For example, science courses may be more 
expensive than English courses. Second, this approach assumes that the average cost of educat- 
ing a student at each college is the same as the average cost of educating students in the study, 
and similarly it assumes that the variation surrounding the actual cost per credit is distributed 
equally among program and control group members. The second row of Table 5.2 presents the 
base cost of credits attempted by students at the six sites. Control group members attempted an 
average of 21.8 credits over the three semesters of follow-up, for a total cost of credits attempt- 
ed (credits attempted multiplied by cost per credit) of $8,214. 6 This represents the average cost 
of educating a student using regular services. 


learning communities, but this may be unnecessary as learning communities become more established. 
However, some research suggests that such payments may be an important factor in ensuring that enough 
motivated instructors and staff are available to implement strong learning community programs (Weissman et 
al., 2009; Asera, 2008). 

5 “Cosf ’ in this case refers to the amount of resources dedicated to the course by the college; it is not neces- 
sarily connected to the price that students may be paying for that course. 

6 See “Calculating the Value of Credits Attempted” in Appendix D for additional detail. 
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The Learning Communities Demonstration 
Table 5.2 

Net Cost of Education per Sample Member (in 2011 Dollars) 


Final Report of the Learning Communities Demonstration 


Type of cost ($) 

Program 

Group 

Control 

Group 

Difference 

(Net) 

Direct cost 3 - cost of program components 

566 

0 

566 

Base cost b - cost of credits attempted in the absence of the program 

8,214 

8,214 

0 

Indirect cost - cost of additional credits attempted due to the program 

138 

0 

138 

Total ($) 

8,919 

8,214 

705 


SOURCES: MDRC calculations from U.S. Department of Education, National Center for Education Statistics, 
Integrated Postsecondary Education Data System (IPEDS) data; MDRC calculations from program- specific 
participation and budget data from the Community College of Baltimore County (CCBC), Houston, and 
Kingsborough. 

NOTES: Costs are calculated based on three semesters of follow-up. 

Rounding may cause slight discrepancies in sums and differences. 

Program costs are based on a steady state of operation that excludes research and start-up costs. 

A very small portion of the program costs can be attributed to control group members who accidentally 
enrolled in learning communities. For practical purposes this analysis assigns zero direct program cost to control 
group members. 

Costs are weighted by number of sample group members. Site-specific values are available in Appendix 
Table E.l. 

The typical cost per credit ($377) is a weighted average based on the sample size of the six college-specific 
cost per credit estimates. Specifically, the estimated cost per credit values are: $484 at CCBC, $301 at 
Hillsborough, $311 at Houston, $393 at Kingsborough, $378 at Merced, and $405 at Queensborough. The 
estimated cost per credit is based on annual expenditures and the annual number of credit hours at each college 
using 2008/2009 values pulled from IPEDS: monetary values are then adjusted for inflation using the Higher 
Education Price Index for public 2-year institutions. Note that these cost per credit estimates are simply proxies 
to estimate average costs within a college. These estimates should not be considered a measure of efficiency and 
should not be compared across colleges, nor do they take into account known marginal costs of learning 
communities. 

“Direct cost is based on data from CCBC, Houston, and Kingsborough. 

b The typical cost of credits attempted is based on data from all six sites. 


Indirect Cost 

Indirect cost describes the cost associated with behavioral changes that are likely a result of 
participating in the learning communities. As Chapter 3 showed, learning communities have an 
impact on the number of credits attempted: Students in the program group attempted more 
credits than students in the control group (see Table 3.1). The bottom row of Table 5.2 shows 
the indirect cost of the program or the cost associated with program group members attempting 
more credits than control group members. Over the three semesters of follow-up, on average, 
each program group member attempted 22.2 credits, while the average control group member 
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attempted 21.8 credits. Multiplying the additional credits attempted by the average cost per 
credit gives an indirect cost of the program of $138. 7 

Net Cost 

The net cost is defined as the difference between the program group cost and the control group 
cost. The costs of each group are presented in the total line of Table 5.2. Adding the direct cost, 
base cost, and indirect cost shows that the total cost of educating the average program group 
member over three semesters was $8,919, while the total cost of educating the average control 
group member was $8,214. Over the three semesters of follow-up, the net cost is around $700 
per program group member. This represents an 8.6 percent increase in cost compared with the 
cost of regular college services without the program. Site-specific net cost values are presented 
in Appendix Table E. I. 8 

Cost-Effectiveness Analysis 

A cost-effectiveness analysis expresses the cost of an intervention as the cost per unit of a 
desired outcome. In this analysis, cost effectiveness considers the cost for any credit earned, 
whether developmental or regular and whether in the targeted subject area or not — the primary 
outcome of this evaluation of learning community programs. Table 5.3 summarizes the results 
of the cost-effectiveness analysis, and Figure 5.1 illustrates how these results were calculated. 

The top row of Table 5.3 presents the total average costs for educating a student in the 
program group and a student in the control group (these values are taken from the preceding 
“Net Cost” section of this chapter). The cost per program group member ($8,919) is $705 more 
than the cost per control group member ($8,214). The second row shows the average number of 
credits earned by program and control group members. The average program group member 
earned 15.6 credits, while the average control group member earned 15.1 credits (this result is 
shown in Table 3.1). The bottom row presents the cost per credit earned for each group: $572 
per credit earned for the program group and $544 per credit earned for the control group. Figure 
5.1 shows how these values are calculated; specifically, it looks at the total cost of each 


7 See “Value of Revenue from Attempting More Credits” in Appendix D for additional detail. 

8 Net cost values in isolation can be deceptive. For example, the least costly learning community program 
was run by CCBC, with a net cost of just over $100 (which represented a 1.1 percent increase in costs over the 
regular college services). It was relatively inexpensive because control group members attempted slightly more 
credits over the follow-up period. This increased control group costs and therefore reduced the net cost of the 
program. 
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The Learning Communities Demonstration 
Table 5.3 


Cost-Effectiveness Values: Pooled Impacts and Costs per Group Member 

(in 2011 Dollars) 

Final Report of the Learning Communities Demonstration 


Outcome 

Program 

Group 

Control 

Group 

Difference 

(Net) 

Cost per group member ($) 

8,919 

8,214 

705 

Number of credits earned per group member 

15.6 

15.1 

0.5 

Cost per credit earned ($) 

572 

544 

29 

Sample size (total = 6,974) 

3,983 

2,991 



SOURCES: MDRC calculations from U.S. Department of Education, National Center for Education Statistics, 
Integrated Postsecondary Education Data System (IPEDS) data; MDRC calculations from the Community College 
of Baltimore County, Hillsborough Community College, Houston Community College, Kingsborough Community 
College, Merced College, and Queensborough Community College transcript data. 

NOTES: Costs are calculated based on three semesters of follow-up. 

Rounding may cause slight discrepancies in sums and differences. 

Estimates are adjusted by round of random assignment. 


group and the total number of credits earned during the follow-up period in order to estimate the 
cost per credit earned. 9 

When the cost to educate program and control group members over the three semesters 
of follow-up is compared with the impact on total credits earned (about half a credit), the typical 
investment in program group members is slightly less effective dollar for dollar than the typical 
investment in control group members. Specifically, the program group cost per credit earned 
($572) is 5.3 percent more than the control group cost per credit earned ($544). This difference 
shows that the impact on credit accumulation was not large enough to make up for the addition- 
al investment in each program group member. In order to be exactly as effective dollar for 
dollar as the regular college services in stand-alone classes, the program would have had to 
operate for $447 less per program group member without negatively affecting the impacts. This 
adjustment requires that more than half of the net cost be cut. Another way of saying this is that 
the typical impact of a learning community of half a credit could justify approximately $258 of 
investment above the cost of the regular college services. 


9 For a more detailed explanation of this calculation, please see “Calculation of Cost per Credit Earned” in 
Appendix D. 
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The Learning Communities Demonstration 
Figure 5.1 

Calculation of the Cost per Credit Earned* by Research Group (in 2011 Dollars) 
Final Report of the Learning Communities Demonstration 


Program firoiin 


$8,919 

cost per student 

X 

3,983 

students 

= 

$35,552,731 
total cost 





- 

15.6 

credits earned 

X 

3,983 

students 

= 

62,049 

total credits earned 



Control Rrono 

$8,214 

cost per student 

X 

2,991 

students 

= 

$24,566,897 
total cost 






15.1 

credits earned 

X 

2,991 

students 

= 

45,177 

total credits earned 


f 

$572 

cost per credit 
earned 

X. J 


f \ 

$544 

cost per credit 
earned 

V- " J 


SOURCES: MDRC calculations from U.S. Department of Education, National Center for Education 
Statistics, Integrated Postsecondary Education Data System (IPEDS) data; MDRC calculations from 
program-specific participation and budget data from the Community College of Baltimore County (CCBC), 
Houston Community College, and Kingsborough Community College; MDRC calculations from CCBC, 
Hillsborough Community College, Houston Community College, Kingsborough Community College, 
Merced College, and Queensborough Community College transcript data. 

NOTES: Rounding may cause slight discrepancies in sums and differences. 

Program costs are based on a steady state of operation that excludes research and start-up costs. 


It is important to keep in mind that failure to operate a learning community at or below 
the cost per desired outcome of the regular college services does not imply that the program’ s 
funding is unjustified. Specifically, the investment in learning communities does positively 
impact key outcomes; it simply costs a bit more per additional outcome, and this may be 
acceptable. Perhaps getting students to earn those additional credits is simply harder than getting 
students to earn the credits they would have earned anyway in the absence of the program, and 
it may be reasonable to pay more for such an impact. Additionally, a college may also choose to 
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operate such a program if it believes that learning communities offer other benefits that are not 
evident in this analysis. 

Conclusion 

The average cost of running a learning community program during the semester of program 
operation is approximately $570 per program group member. When the additional costs of 
educating students are considered, over three semesters of follow up, colleges invested on 
average just over $700 more per program group member than it did per control group member 
(an 8.6 percent increase over the base investment made in control group members). This 
investment is slightly less cost-effective at getting students to earn credits than the regular 
college services, but in general the intervention does get participants to earn on average about 
half a credit more than they would otherwise. Decision makers considering an investment in a 
learning community program will need to weigh the impact of learning communities against the 
net costs along with other factors, such as faculty interest, when deciding whether to invest in 
this strategy. 


76 



Chapter 6 

Conclusions and Implications 


Learning communities have become a popular intervention used by community colleges that are 
trying to improve the educational trajectories of the increasing number of students who are 
referred to developmental education. This report addresses the question of whether this inter- 
vention leads to better outcomes for these academically underprepared students than traditional 
developmental education — or “business as usual.” With many studies now documenting the 
very low likelihood that students in traditional developmental education will progress to 
college-level courses, and the even lower odds that they will earn a credential, the pressure on 
colleges to find effective, scalable solutions remains intense. 

The predominant model in developmental education in community colleges is a one- 
semester learning community offered in the first or second semester of the freshman year that 
links a developmental course with another developmental course, a college-level course, or a 
“student success course.” One such program became the subject of a rigorous evaluation 
conducted by MDRC as part of the Opening Doors demonstration. The results were promising: 
Students in learning communities appeared to progress more quickly through the developmental 
English course sequence than a control group who could enroll in traditional, stand-alone 
developmental courses. Building on these results, within a few years the Learning Communities 
Demonstration, a project of the National Center for Postsecondary Research, was launched to 
test a range of one-semester learning community programs as typically offered by community 
colleges to students who are referred to developmental course work in English or mathematics. 

Six Experimental Evaluations of Learning Community Programs 

To date, a total of six randomized controlled trials have been conducted in six colleges to 
estimate the impacts of learning communities on educational outcomes for developmental 
students. Five of the colleges were part of the Learning Communities Demonstration and one 
was the Opening Doors learning community program at Kingsborough Community College. 
(The Opening Doors program served students who were assessed as “college-ready” in English, 
in addition to students who were referred to developmental English.) The site-specific findings 
on program impacts from the five Learning Communities Demonstration programs were less 
encouraging than those from the Opening Doors study. Up to two semesters after the program 
ended, short-term impacts on credits earned in the subject targeted by the learning community 
were discerned in most of the demonstration colleges, but these faded after a semester or two. 
None of the learning community programs had an effect on reenrollment (or persistence) in 
college. 
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This report pools data across all six sites to examine the average effects of learning 
communities on outcomes observed over three semesters (including the semester in which 
students in the program group enrolled in a learning community). For more detailed accounts of 
the implementation and impacts of the individual programs, including the results of a longer 
follow-up of the Kingsborough sample, readers are encouraged to refer to the publications listed 
at the end of this report. 

A Brief Summary of the Key Findings of This Report 

Overall, the analysis in this report reinforces the key findings presented in the previously 
published site-specific reports. On average, one-semester learning communities as typically 
operated in community colleges for developmental students have a positive but small estimated 
impact (half a credit) on credit accumulation. This impact is achieved primarily in the semester 
in which students are enrolled in the learning communities and is driven largely by credits 
earned in the targeted subject area. The analysis found no impact on persistence up to three 
semesters after students enrolled in the study. 

In addition to the findings for these key outcomes, results from an analysis of variation 
in the programs’ effects across the six sites and for selected subgroups of students are presented. 
These findings suggest that the impacts on credits earned in the targeted subject varied signifi- 
cantly across the six colleges. The Kingsborough program produced an estimated impact that 
was one credit larger than the average estimated impact for this outcome. However, there was 
not strong evidence of variation in the programs’ impacts on total credits earned across the sites. 
The analyses showed no meaningful variation in impacts by race and gender of the study 
students, time between graduation from high school and college enrollment, and earlier versus 
later cohorts of students in the study. 

Additionally, the report reviews the implementation studies at each site to discern any 
patterns that could help explain the relatively modest average impacts. Two possible explana- 
tions are explored. The first is the possibility that the programs were not implemented as 
intended by the colleges. The second is that the control group received similar services or had 
classroom experiences that were similar to those of the program group. Neither explanation for 
the modest impacts is supported consistently by the implementation research. However, 
considerable variation in the extent to which curricula and content were tightly linked in 
individual learning communities was observed in all six sites. In fact, no more than a handful of 
learning communities at each college “met the standard” for this component of the learning 
community model. This study does not address the question of whether such learning communi- 
ties would have produced larger impacts. 
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The Different Pattern of Impacts of the Kingsborough Program 

One of the most interesting results of this report is that the estimated impacts of the program at 
Kingsborough on credits earned in the targeted subject were larger than those of the other five 
programs. An MDRC report on the Opening Doors Learning Communities program at Kings- 
borough shows that these early impacts were sustained well beyond the three semesters of 
follow-up reported in this publication . 1 Certain differences in the setting, the program, and the 
types of students served at Kingsborough were explored to shed light on why students seemed 
to have benefitted more from learning communities than at the other colleges. Some of the 
possible differences that could account for the difference in impacts are: 

• The program model included more enhancements than the other models, par- 
ticularly in the area of student support services. 

• The Kingsborough learning communities were associated with a higher 
number of credits than the other programs. Also, students in the program 
were actively encouraged by their advisers to enroll in six-week interses- 
sions, further increasing the number of credits students could earn in the first 
year of college. 

• Students in the Kingsborough learning community program were more likely 
to be full time and financially dependent than students in the programs at the 
other colleges. 

• The Opening Doors program enjoyed unusually strong support from college 
administrators, from the president down to department chairs, surpassing that 
of any of the other colleges. 

Three Ways to Place the Impact Findings in Context 

The main finding of this report is that, in general, developmental education learning communi- 
ties can be expected to produce an impact of about half a credit. But is a half-credit gain, 
especially when the credit is most likely a developmental credit, a meaningful gain? There are 
several ways to answer this question. 

First, the impact of half a credit can be compared with the impacts of other similar in- 
terventions with similar goals. Several recent randomized controlled trials of new approaches to 
serving developmental education students have found no or modest impacts on credit accumula- 


Som mo, Mayer, Rudd, and Cullinan (2012). 
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tion. 2 But some programs have been shown to have considerably larger impacts. 3 For example, 
a randomized study of Accelerated Study in Associate Programs (ASAP), which targeted 
students at three City University of New York colleges who were in need of one or two devel- 
opmental education courses, found that students assigned to a program group earned 4.5 more 
credits over two semesters than their counterparts in a control group. 4 

A second way to put the impact findings in this report in context is to assess the likeli- 
hood that a half-credit gain in the first three semesters will make a difference in the odds of 
completing college and earning a credential. After three semesters in the study, students in the 
six learning community programs in this report earned an average of only a little more than 15 
credits (there was no significant difference between program and control groups). Since an 
associate’ s degree requires that a student earn 60 nondevelopmental credits, these students, on 
average, were not even one -quarter of the way toward an associate’s degree after three semes- 
ters. Further, students passed on average only about 70 percent of the credits they attempted. 
With this pass rate, a student would have to take almost 90 credits in order to earn enough 
credits to graduate, not counting developmental credits that are typically prerequisites for 
enrolling in courses that count toward a degree. For a full-time student attempting 15 credits per 
semester, this would take at least six semesters. The half-credit boost learning communities can 
produce, compared with the total number of credits needed, is not likely to change the odds of 
earning a degree. 

A final and perhaps the most important piece of context is the cost of learning commu- 
nities. If the additional cost of running learning communities is negligible, than a half-credit 
gain may be worth the effort. Chapter 5 reported estimated costs for three of the six studied 
programs, one of which was the more advanced Kingsborough program. The average cost for 
these three programs was about $570 per student over and above the regular costs of serving 
developmental education students. Colleges need to weigh the relative cost of learning commu- 
nities against the probable gain of half a credit along with considerations of other benefits for 
the institution or faculty. Some of those benefits are described in the next section. 


2 Richburg-Hayes, Sommo, and Welbeck (2011); Visher, Butcher, and Cema (2010); Zachry Rutschow, 
Cullinan, and Welbeck (2012). 

Additional programs targeting academically underprepared community college students that were found 
to have large effects on credits earned and other outcomes include Accelerated Study in Associate Programs 
(ASAP), the Accelerated Learning Program, and Integrated Basic Skills Education (I-BEST). See Scrivener, 
Weiss, and Sommo (2012); Jenkins et al. (2010); Zeidenberg, Cho, and Jenkins (2010). 

4 Scrivener, Weiss, and Sommo (2012). 
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Lessons Learned for Students, Instructors, and Decision Makers 
in Community Colleges 

Learning communities have a tendency to elicit strong reactions, mostly positive, from many of 
the students, instructors, and college leaders who operate, teach, or enroll in them. The findings 
in this report should not diminish enthusiasm for the model among its supporters but will 
hopefully inform strategic decision making by community colleges as they struggle to “move 
the needle” on academic outcomes for this population. Below are some of the lessons learned 
from this research, based on the experiences of students, instructors, and college leaders. 

From the Students’ Perspective: 

Learning communities tend to be a positive socioemotional experience for most stu- 
dents. Findings from a survey of students in the Opening Doors learning communities program 
at Kingsborough as well as from student focus groups conducted at all six colleges showed that 
many students tended to report that they felt a heightened sense of belonging to the class or 
college community and had gotten to know other students and their instructors better or faster. 
Some reported feeling very supported by these relationships both personally and academically . 5 
Learning communities also encourage students to take certain courses they need to graduate but 
that they might otherwise avoid or postpone, such as developmental courses or student success 
courses. However, students should not expect one-semester learning communities alone to help 
them overcome the obstacles they may face in achieving their educational goals. 

From the Instructors’ Perspective: 

Instructors almost invariably report that learning communities are a lot of work and re- 
quire more preparation and planning time than traditional, stand-alone courses. Not all instruc- 
tors are suited for or enjoy the collaborative nature of teaching in learning communities. But for 
those who do, learning communities can be so rewarding that many instructors insist they 
would teach no other way. Benefits include closer relationships with students, stronger collegial 
relationships and professional community, and the opportunity to use different instructional 
strategies. 

From the College Administrators’ Perspective: 

One-semester learning communities do no harm to student achievement and may offer 
other benefits. Some colleges report that learning communities can be a useful tool to motivate 
or train faculty. And some instructors report that their teaching improves as a result of being 


5 Scrivener et al. (2008); Visher, Schneider, Wathington, and Collado (2010). 
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involved in learning communities. Colleges may want to run learning communities with 
enhancements such as strong support components and additional advising, since these may lead 
to better impacts. But bringing such programs to scale requires effort, resources, and time. 

Limitations of This Study and the Need for Future Research 

The research described in this report, although significantly advancing the amount of evidence 
available to inform policy and practice on learning communities, was not designed to address 
every question of interest about learning communities. There are several limitations to the study, 
the most important of which are listed here as possible avenues for future research. 

First, the outcomes measured in this study focus on persistence and progress toward de- 
gree completion primarily measured as number of credits earned. Because resources were 
constrained, no attempt was made to measure the impact of the program on learning outcomes 
or learning skills. Proponents of learning communities argue that integrated curricula and active, 
collaborative learning in learning communities may lead to development of higher-order 
cognitive skills and deeper mastery of the content. Future research should include measures of 
learning assessed, for example, through the use of a standardized assessment instrument. 

Another set of outcomes not measured in this research is the effect of learning commu- 
nities on institutional change. Colleges often report that operating learning community programs 
results in improvements in processes such as registration and collaboration between the aca- 
demic and student services divisions . 6 For example, several colleges in the demonstration 
reported that solving the logistical challenges of scheduling cohorts of students in linked, block- 
scheduled classes led to more opportunities for useful one-on-one advising with students as well 
as more user-friendly registration systems. Learning communities also turned out to be useful 
for promoting faculty development. More attention to these effects of learning communities 
may be warranted. 

Additionally, although the implementation research revealed significant variation 
among learning communities offered within each college in the study, no attempt was made to 
measure either the quality of each learning community or the relationship between quality and 
impacts. To do so would have required formal observation of the teaching and learning in 
learning community classrooms and was therefore well beyond the scope of this study. Howev- 
er, the research helped to underscore the urgency of conducting rigorous research on what goes 
on inside the classroom in contemporary community colleges. Classroom observation and 
assessment as well as measurements of teacher quality are needed to advance knowledge about 
how to improve student outcomes through more effective teaching. 


6 Weissman et al. (2009). 
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A final limitation of the study is the relatively short time students were tracked after 
random assignment. While three semesters may be long enough to discern impacts for most 
learning community programs, it proved to be too short to observe the long-term impacts on 
reenrollment and completion in the Kingsborough program. Unfortunately, resources did not 
permit longer follow-up of students in the studies at the other sites, so it remains unknown if 
results similar to those found for the Kingsborough program would have been observed several 
years later at any of the other colleges. However, it seems unlikely that longer follow-up would 
have captured effects beyond the second year in those programs where impacts did not last 
beyond the first semester. 

Summing Up 

Despite these limitations, colleges and policymakers now have available a relatively large and 
diverse amount of evidence to consider while making strategic choices about whether or how to 
invest in learning communities. The evidence, described here and in the other publications listed 
at the end of this report, provides information on a range of topics, including the implementation 
experience, the challenges of scaling up learning communities, the impacts on educational 
outcomes up to six years after participation in the program, whether the program works better 
for some students than for others, and the costs of learning communities relative to their 
effectiveness. 

The overall conclusion from these reports is that learning communities as typically op- 
erated in community colleges, on average, should not be expected to produce more than a 
modest impact on credits earned and that that this intervention, by itself, will not likely lead to 
higher rates of reenrollment and completion for academically underprepared students. However, 
the evidence also suggests that a learning community program with substantially enhanced 
supports for students, such as ongoing or extra advising and the opportunity to accumulate more 
credits early, may lead to greater benefits than the average learning community program. The 
Opening Doors program at Kingsborough resulted in more credits earned in the targeted subject 
area than the other programs, and these promising short-term impacts grew into long-term 
impacts. As discussed in an MDRC report on Kingsborough’ s Opening Doors Learning 
Communities program, in the six years after the learning community experience at Kings- 
borough, learning community students consistently outperformed the control group in credits 
earned and were more likely to graduate . 7 


7 Sommo, Mayer, Rudd, and Cullman (2012). The program at Kingsborough was associated with the 
greatest increases in graduation for students who were not in need of developmental English at the start of the 
study. 
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Policymakers and colleges need to consider all of this evidence when deciding whether 
to run learning communities, how much to invest in them, how to design them, and how to 
ensure that they are implemented as intended. Above all, they should have realistic expectations 
of what a one-semester intervention of any kind can do to change the trajectories of large 
numbers of academically underprepared students. The road to a credential is a long one that 
starts on the day that new would-be students receive the news that they failed one or more 
placement tests and must enroll in developmental education. The research on learning commu- 
nities suggests that most of these programs as typically operated in today’s community colleges 
are not likely to make the journey shorter. 
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Appendix A 1 

Impact Analyses 


'Parts of this appendix were published in Weiss, Visher, and Wathington (2010). 




Introduction 


This technical appendix provides additional information relevant to the impact analyses pre- 
sented in this report. It begins with an overview of some key study design features that influence 
the impact analyses. Next, it describes how the statistical procedure was selected, given some 
issues that arise in an individually randomized group treatment trial. Finally, the pooled impact 
model and the models used to estimate variation in impacts are described, illustrated with 
sample SAS code. 


Study Design 

Several features of the evaluation designs influenced analysis decisions. First, at three of the six 
colleges, more than one campus took part in the study, resulting in a total of 1 1 campuses. 
Second, at each college more than one “cohort” of students took part in the study. A cohort 
refers to a group of students entering the study at the beginning of a specific semester (for 
example, Hillsborough had a fall 2007, spring 2008, and fall 2008 cohort). In total, there were 
23 unique college-by-cohort combinations, with Hillsborough having three cohorts and the 
remaining five colleges each having four cohorts. There were 36 unique campus-by-cohort 
combinations. Third, individual students were randomly assigned to the program or control 
group, and this random assignment was conducted separately for each unique campus-by-cohort 
combination, amounting to what can be thought of as 36 unique lotteries, or 36 distinctive 
random assignment blocks. Appendix Table A.l lists the colleges and campuses, including the 
number of cohorts at each campus. These design features are relevant when choosing covariates 
in the impact models described in a later section of this appendix. 


Choosing an Impact Model 

Within each campus-by-cohort group, most program group students clustered into learning 
communities after they were randomly assigned to the opportunity to participate in a learning 
community. In order to account for the clustering of students in learning communities, SAS’s 
PROC SURVEYREG was used. Below is a description of why this procedure was selected, 
rather than the common alternative, PROC MIXED. 

Many evaluations of education interventions are designed so that clusters or groups 
of students, rather than individual students, are randomly assigned to a program or control 
group. The study design in the evaluations described in this report is different in that 
randomization occurred at the student level; however, most students in the program group 
were clustered into learning communities, while students in the control group were essen- 
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The Learning Communities Demonstration 
Table A.l 

Campus by Cohort Combinations 

Final Report of the Learning Communities 
Demonstration 


College 

No. of 
Cohorts by 
College 

Campus 

No. of 
Cohorts by 
Campus 

CCBC 

4 

Catonsville 

4 



Essex 

4 

Hillsborough 

3 

Dale Mabry 

3 



Ybor City 

3 



Brandon 

1 

Houston 

4 

Northline 

4 



Southeast 

3 



Central 

2 

Kingsborough 

4 

- 

4 

Merced 

4 

- 

4 

Queensborough 

4 

- 

4 

Total 

23 


36 


SOURCES: Scrivener et al. (2008); Weiss, Visher, and 
Wathington (2010); Weissman et al. (2011); Weissman et al. 
( 2012 ). 


tially unclustered . 2 This partially clustered design (or “partially nested design,” as it is 
sometimes referred to) needs to be accounted for in the analyses because the clustered 
program group students’ outcomes may not be independent, violating the assumption of 
independence of errors in ordinary least squares regression. However, it may not be appro- 
priate to use the typically formulated hierarchical linear model used in many cluster- 
randomized trials because the control group is comprised of what can be thought of as many 


2 Control group students were free to enroll in courses of their choosing as long as they were not part of a 
learning community. It is likely that on occasion multiple control group students were in the same class. Less 
common (but still possible), some control group students may have been in multiple classes together. However, 
in all analyses, control group students are treated as though they are in independent clusters of size one. 
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independent clusters of size one. As such, models that explicitly account for the partially 
clustered design have been suggested/ Such a model can be written as: 

Equation A.1: 

Level 1: y tj = /3 0j + 

Level 2: /3 0j = y 00 

Ptj — Tio + u ij 

Where y t j is an academic outcome (for example, credits earned) for student i in learn- 
ing community j, Tij is a binary indicator of treatment status for student i in learning community 
j. In addition, is the student-level residual and u 1; - allows for differences across clusters in 
the treatment effect. 3 4 

This model implies that the intercept is constant (has no additional source of variation) 
and represents the average outcome for the control group. In contrast, the second part of the 
level 2 equation shows that the program group has a mean difference of y 10 from the control 
group average, and this difference is allowed to vary across clusters. This seems plausible for 
the learning communities intervention: The control group student-level variation and group- 
level variation cannot be separated, while program group students have an average difference 
from the control group mean that is allowed to vary both because of student-level differences 
and because of variation associated with the learning community in which they enrolled. The 
average learning community impact could vary because the efficacy of some learning commu- 
nity links vary, or the instructors teaching the links vary in effectiveness, or the particular mix of 
students in the linked classes induce a positive (or negative) effect. Such a model can be fit 
using a software procedure like SAS’s PROC MIXED. 

However, the analyses presented in this report are further complicated by nonparticipa- 
tion. The program group is not fully nested, since around 29 percent of program group students 
did not enroll in a learning community. As such, many program group students were in clusters 
of size one, whereas the remaining program group students were in clusters whose size was 

3 Bauer, Sterba, and Hallfors (2008). 

4 This model is described in detail in Bauer, Sterba, and Hallfors (2008). 
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often between 20 and 30. Moreover, the outcomes of interest (credits earned, continued registra- 
tion) were related to cluster size, since over half of the program group students who were in 
clusters of size one did not register for any courses. Thus, their mean outcomes are much lower 
than the rest of the program group. Essentially, the program group can be thought of as coming 
from two different populations: one population of larger clusters with relatively high means and 
another population of clusters of size one with relatively low means. In this circumstance, using 
a statistical procedure like SAS’s PROC MIXED yields point estimates that are inappropriate, 
given a study design where each individual should contribute approximately equally to point 
estimates . 5 Point estimates derived using PROC MIXED are driven toward the mean of the 
clusters of size one (that is, individuals in clusters of size one have more weight than individuals 
in larger clusters). Such analyses are inappropriate given the design of this study. 

An alternative statistical procedure that can be used with this data structure is SAS’s 
PROC SURVEYREG. 6 This procedure allows users to identify clusters in order to account for 
the potential lack of independence of observations within clusters (that is, it uses cluster-robust 
standard errors). In addition, point estimates derived using this procedure are identical to those 
using a general linear model (for example, using SAS’s PROC GLM). MDRC conducted 
analyses on simulated data 7 and found that this procedure produced the expected point estimates 
(given the data-generating mechanism) and slightly conservative standard errors (that is, 
standard errors tended to be slightly larger than expected). Consequently, this procedure was 
used in all impact analyses. 

These simulations and selected procedure do not consider that the mechanism by which 
students sort into learning communities may produce the appearance of lack of independence in 
observations, unrelated to students’ experiences. The authors believe that the post-random 
assignment sorting process alone is not a true source of lack of independence and thus should 
not influence the standard error of the outcome; however, the authors are unaware of any 
procedures that are able to account for this issue. While this is an important topic for future 
methodological research, the current approach is likely adequate and is most likely a bit con- 
servative. 


5 MDRC’s internal simulations tested and demonstrated this point. Please contact the authors for more 
information. 

6 See SAS Institute, Inc. (2012a) for more information on how this procedure calculates regression coeffi- 
cients and standard errors. 

7 Several data-generating mechanisms were tested. The mechanisms were intentionally designed to create 
data that were fairly similar to the observed data in this study. 
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The Pooled Impact Model 

The previous section discussed the approach used to account for the clustering of program 
group students into learning communities. Described here are variables included in the model 
used to obtain the pooled impact estimate. For notational simplicity, the potential lack of 
independence of errors caused by clustering is not explicitly expressed in each equation’ s error 
term. Suffice it to say that the errors are not treated as independent and identically distributed 
normal — the standard errors are cluster-robust as calculated by SAS’s PROC SURVEYREG. 

The pooled model produces estimates of the average effect of the opportunity to partic- 
ipate in learning communities. The equation in the model specifies an outcome measure (for 
example, total credits earned) as a function of indicator variables that identify each random 
assignment block and a single treatment indicator that distinguishes between sample members 
randomly assigned to the program group and control group. 

As noted above, random assignment was conducted separately for each cohort of stu- 
dents within each campus in the study. The random assignment block indicators identify each 
unique cohort-by-campus combination. In total, there are 36 block dummies. 8 

Equation A.2: 

y = ^ ^ a kl RA_Block kl + p 0 t + e 

l k 

where: 

y — an outcome for a student 

RA_Block kl = a cohort-by-campus dummy indicator equal to one if the student is 
from cohort k and campus l, and zero otherwise. There are 36 
unique cohort x campus dummy variables. 

t = a treatment indicator equal to 1 if the student was randomly as- 

signed to the program group, and 0 otherwise. 

The estimated value of the treatment indicator coefficient (/? 0 ) is the estimated average 
effect of the opportunity to participate in learning communities. Tables and figures in this report 

8 Weights were used to account for varying random assignment ratios within college. Within each of the 
six colleges, the weights sum to the colleges’ total sample size so that the pooled impact estimates are roughly 
weighted by the colleges’ sample size. Weights were obtained separately for each college and created to ensure 
that across blocks within colleges the random assignment ratio is constant. 


91 



present the least squares means for program and control group members. 9 Sample SAS code is 
provided below: 


PROC SURVEYREG DATA = SET1 ; 

CLASS E CAMPUS REL_COH ; 

MODEL R1T0TCR = E CAMPUS *REL_COH / SOLUTION ; 
WEIGHT WGTS ; 

CLUSTER LCLINK ; 

LSMEANS E / PD IFF = CONTROL OM ; 

RUN ; 


Estimating Variation in Impacts 

The previous section described the model used to obtain the pooled program effect. Described 
here are the models used to determine whether there is evidence of variation in impact estimates 
across colleges and cohorts. 

Impact Variation by College 

Equation A.3 describes a model that produces estimates of the average effect of the 
opportunity to participate in learning communities at each college. 

Equation A.3: 

y-II a kl RA_Block kl + ^ P m (t * College m ) + e 

l k m 

where: 

College m = a dummy indicator equal to one if the student was randomly as- 
signed at college m, and zero otherwise. 

The estimated value of the treatment indicator coefficients (/? m ’s) is the estimated aver- 
age effect of the opportunity to participate in learning communities for college m's sample. 
When testing for evidence of variation in impacts among the six colleges, the null hypothesis is 
that the /? m ’s are all equal to each other. More formally: 


9 For more details, please refer to SAS Institute, Inc. (2012b). All LS MEANS calculations use the OM 
option. 
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H 0 : /?1 — /?2 — /?3 — /?4 — /?5 — f>6 


If the null hypothesis is rejected, it means that there is evidence that the impacts 
vary across the colleges. The joint-F test used to assess impact variation is essentially 
testing to see whether there is any added predictive power to including the college-by- 
treatment interaction terms in the model compared with just including a single treatment 
indicator as in Equation A.2. 


PROC SURVEYREG DATA = SET1 ; 

CLASS E SITE CAMPUS REL_COH ; 

MODEL R1TOTCR = CAMPUS *REL_COH E SITE*E / NOINT SOLUTION ; 
WEIGHT WGTS ; 

CLUSTER LCLINK ; 

LSMEANS E / PD IFF = CONTROL OM ; 

RUN; 


Impact Variation by Cohort 

To assess variation in impacts across cohorts, it is most interesting to determine 
whether impacts tend to vary across cohorts within colleges, rather than overall (which 
could be influenced by variation in impacts among colleges). As a result, college-by-college 
impacts were controlled for in these analyses. The model is described by the following 
equation: 

Equation A.4: 


y = 



a kl RA_Block kl 


m k 


Cohort k ) + e 


The joint-F test used to assess impact variation across cohort is essentially testing to see 
whether there is any added predictive power to including the college-by-treatment-by- 
cohort interaction terms in the model compared with just including a college -by-treatment 
indicator as in Equation A.3. 


PROC SURVEYREG DATA = SET1 ; 

CLASS E SITE CAMPUS REL_COH ; 

MODEL R1T0TCR = CAMPUS *REL_COH E SITE*E SITE*REL_COH*E / 
NOINT SOLUTION ; 

WEIGHT WGTS ; 

CLUSTER LCLINK ; 

LSMEANS E / PD IFF = CONTROL OM ; 

RUN; 
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Similar procedures were used for the subgroup analyses that identified subgroups based on 
individual characteristics, such as whether a study participant was a recent high school 
graduate. 
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Appendix B 

Supplementary Exhibits for Chapter 3 




The Learning Communities Demonstration 
Table B.l 

Pooled Academic Outcomes: Developmental and College-Level Credits 
Final Report of the Learning Communities Demonstration 


Outcome 

Program 

Group 

Control 

Group 

Difference 

(Impact) 

Standard 

Error 

Develoumental credits 

Cumulative credits attempted 

First semester 

5.57 

5.11 

0.46 *** 

0.15 

Second semester 

7.57 

7.17 

0.40 * 

0.22 

Third semester 

8.39 

8.10 

0.29 

0.24 

Cumulative credits earned 

First semester 

3.55 

3.09 

0.45 *** 

0.14 

Second semester 

5.04 

4.64 

0.40 ** 

0.20 

Third semester 

5.65 

5.30 

0.35 * 

0.21 

Colleee-level credits 

Cumulative credits attempted 

First semester 

4.60 

4.77 

-0.17 

0.12 

Second semester 

9.41 

9.43 

-0.02 

0.24 

Third semester 

13.77 

13.70 

0.07 

0.36 

Cumulative credits earned 

First semester 

3.35 

3.29 

0.07 

0.11 

Second semester 

6.87 

6.72 

0.15 

0.22 

Third semester 

9.90 

9.80 

0.10 

0.31 

Total credits 

Cumulative credits attempted 

First semester 

10.18 

9.88 

0.29 * 

0.18 

Second semester 

16.99 

16.60 

0.38 

0.31 

Third semester 

22.18 

21.81 

0.37 

0.43 

Cumulative credits earned 

First semester 

6.91 

6.38 

0.53 *** 

0.17 

Second semester 

11.92 

11.36 

0.56 * 

0.30 

Third semester 

15.58 

15.10 

0.47 

0.39 

Sample size (total = 6,974) 

3,983 

2,991 




(continued) 
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Table B.l (continued) 


SOURCE: MDRC calculations from the Community College of Baltimore County, Hillsborough Community 
College, Houston Community College, Kingsborough Community College, Merced College, and Queensborough 
Community College transcript data. 

NOTES: Rounding may cause slight discrepancies in sums and differences. Targeted subject credit measures and 
outside the targeted subject credit measures may not sum to total credit measures because Hillsborough Community 
College includes vocational credits in total credit measures. These vocational credits are not included in targeted 
subject credit measures or in outside the targeted subject credit measures. Cumulative measures exclude repeated 
courses. 

A two-tailed t-test was applied to differences between research groups. Statistical significance levels are 
indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 

The probability of being assigned to the treatment group varies across colleges and within random assignment 
cohorts, and estimates are weighted to account for the different random assignment ratios. Estimates are adjusted 
by campus and cohort. Standard errors are clustered by learning community link. 
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The Learning Communities Demonstration 
Table B.2 

The Community College of Baltimore County: Academic Outcomes 
Final Report of the Learning Communities Demonstration 


Outcome 

Program 

Group 

Control 

Group 

Difference 

(Impact) 

Standard 

Error 

Enrollment ( % ) 

Enrollment rate 11 

First semester 

84.5 

84.6 

- 0.1 

2.2 

Second semester 

64.2 

64.6 

-0.5 

2.9 

Third semester 

49.9 

49.4 

0.5 

3.1 

Full-time registration b 

First semester 

59.6 

61.2 

- 1.6 

3.0 

Second semester 

39.1 

43.6 

-4.5 

3.0 

Third semester 

28.3 

27.4 

0.9 

2.7 

Enrolled in a learning community 

First semester 

74.2 

0.5 

73 7 *** 

1.8 

In the tarseted subiect c 

Cumulative credits attempted 

First semester 

3.99 

4.13 

-0.14 

0.15 

Second semester 

5.38 

5.58 

- 0.20 

0.19 

Third semester 

6.16 

6.37 

- 0.21 

0.22 

Cumulative credits earned 

First semester 

2.42 

2.53 

- 0.11 

0.15 

Second semester 

3.64 

3.83 

- 0.20 

0.22 

Third semester 

4.32 

4.56 

-0.25 

0.25 

Completed targeted developmental course sequence (%) 

Cumulative third semester 

56.8 

60.5 

-3.7 

3.0 

Outside the tarseted subiect 

Cumulative credits attempted 

First semester 

5.52 

5.58 

-0.06 

0.20 

Second semester 

9.95 

10.55 

-0.60 

0.41 

Third semester 

13.73 

14.25 

-0.52 

0.62 

Cumulative credits earned 

First semester 

3.43 

3.30 

0.14 

0.20 

Second semester 

6.41 

6.55 

-0.14 

0.41 

Third semester 

9.11 

9.33 

- 0.21 

0.60 


(continued) 
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Table B.2 (continued) 


Outcome 

Program 

Group 

Control 

Group 

Difference 

(Impact) 

Standard 

Error 

Total credits 

Cumulative credits attempted 

First semester 

9.52 

9.71 

-0.19 

0.29 

Second semester 

15.33 

16.13 

-0.80 

0.54 

Third semester 

19.89 

20.62 

-0.73 

0.77 

Cumulative credits earned 

First semester 

5.85 

5.83 

0.02 

0.31 

Second semester 

10.04 

10.38 

-0.34 

0.57 

Third semester 

13.43 

13.89 

-0.46 

0.80 

Sample size (total = 1,083) 

650 

433 




SOURCE: MDRC calculations from the Community College of Baltimore County (CCBC) transcript data. 

NOTES: Rounding may cause slight discrepancies in sums and differences. Cumulative measures exclude 
repeated courses. 

A two-tailed t-test was applied to differences between research groups. Statistical significance levels are 
indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 

The probability of being assigned to the treatment group varies across colleges and within random assignment 
cohorts, and estimates are weighted to account for the different random assignment ratios. Estimates are adjusted 
by campus and cohort. Standard errors are clustered by learning community link. 

“Enrollment is based on courses that sample members are still enrolled in at the end of the add/drop period. 
•“Students are registered full time if they attempt 12 or more credits in a semester. 

“Courses in the targeted subject at the CCBC are English courses and include both college-level and 
developmental courses. 
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Table B.3 

Hillsborough Community College: Academic Outcomes 
Final Report of the Learning Communities Demonstration 


Outcome 

Program 

Group 

Control 

Group 

Difference 

(Impact) 

Standard 

Error 

Enrollment ( % ) 

Enrollment rate 3 

First semester 

82.0 

83.4 

-1.4 

3.3 

Second semester 

60.0 

54.7 

5.3 

3.5 

Third semester 

45.9 

46.4 

-0.5 

3.4 

Full-time registration b 

First semester 

47.7 

48.8 

-1.1 

3.8 

Second semester 

33.4 

31.9 

1.5 

3.3 

Third semester 

22.0 

24.2 

-2.2 

2.7 

Enrolled in a learning community 

First semester 

69.0 

0.4 

68.6 *** 

4.3 

In the targeted subiect c 

Cumulative credits attempted 

First semester 

4.65 

4.96 

-0.31 

0.26 

Second semester 

7.36 

7.59 

-0.24 

0.42 

Third semester 

8.45 

8.91 

-0.46 

0.46 

Cumulative credits earned 

First semester 

3.51 

3.71 

-0.20 

0.25 

Second semester 

5.55 

5.82 

-0.26 

0.39 

Third semester 

6.23 

6.52 

-0.29 

0.42 

Completed targeted developmental course sequence (%) 

Cumulative third semester 

50.6 

49.2 

1.4 

4.3 

Outside the targeted subiect 

Cumulative credits attempted 

First semester 

4.57 

4.29 

0.28 

0.24 

Second semester 

8.54 

7.96 

0.58 

0.51 

Third semester 

12.22 

11.54 

0.68 

0.74 

Cumulative credits earned 

First semester 

3.19 

2.78 

0.42 ** 

0.21 

Second semester 

5.86 

5.19 

0.67 

0.41 

Third semester 

7.59 

6.84 

0.75 

0.57 


(continued) 
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Table B.3 (continued) 


Outcome 

Program 

Group 

Control 

Group 

Difference 

(Impact) 

Standard 

Error 

Total credits 

Cumulative credits attempted 

First semester 

9.26 

9.25 

0.01 

0.42 

Second semester 

15.97 

15.55 

0.43 

0.77 

Third semester 

20.83 

20.54 

0.28 

1.04 

Cumulative credits earned 

First semester 

6.74 

6.49 

0.25 

0.41 

Second semester 

11.49 

11.01 

0.49 

0.70 

Third semester 

13.98 

13.37 

0.61 

0.87 

Sample size (total = 1,071) 

709 

362 




SOURCE: MDRC calculations from Hillsborough Community College transcript data. 

NOTES: Rounding may cause slight discrepancies in sums and differences. Targeted subject credit measures and 
outside the targeted subject credit measures may not sum to total credit measures because vocational credits are 
included in total credit measures. These vocational credits are not included in targeted subject credit measures or 
in outside the targeted subject credit measures. Cumulative measures exclude repeated courses. 

A two-tailed t-test was applied to differences between research groups. Statistical significance levels are 
indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 

The probability of being assigned to the treatment group varies across colleges and within random assignment 
cohorts, and estimates are weighted to account for the different random assignment ratios. Estimates are adjusted 
by campus and cohort. Standard errors are clustered by learning community link. 

“Enrollment is based on courses that sample members are still enrolled in at the end of the add/drop period. 

b Students are registered full time if they attempt 12 or more credits in a semester. 

“Courses in the targeted subject at Hillsborough Community College are English courses and include both 
college-level and developmental courses. 
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The Learning Communities Demonstration 
Table B.4 

Houston Community College: Academic Outcomes 
Final Report of the Learning Communities Demonstration 


Outcome 

Program 

Group 

Control 

Group 

Difference 

(Impact) 

Standard 

Error 

Enrollment ( % ) 

Enrollment rate 11 

First semester 

84.3 

81.2 

3.1 

2.9 

Second semester 

61.0 

60.9 

0.1 

3.0 

Third semester 

44.8 

47.4 

-2.6 

3.1 

Full-time registration b 

First semester 

17.3 

20.7 

-3.4 

2.4 

Second semester 

26.3 

25.0 

1.3 

2.7 

Third semester 

19.6 

21.8 

-2.1 

2.4 

Enrolled in a learning community 

First semester 

71.0 

1.5 

69.5 *** 

3.7 

In the tarseted subiect c 

Cumulative credits attempted 

First semester 

2.44 

2.18 

0.26 ** 

0.10 

Second semester 

3.77 

3.46 

0.31 * 

0.16 

Third semester 

4.45 

4.09 

0.36 * 

0.20 

Cumulative credits earned 

First semester 

1.69 

1.26 

0.43 *** 

0.11 

Second semester 

2.33 

2.00 

0.33 ** 

0.15 

Third semester 

2.80 

2.55 

0.25 

0.18 

Completed targeted developmental course sequence (%) 

Cumulative third semester 

9.2 

9.0 

0.2 

1.8 

Outside the tarseted subiect 

Cumulative credits attempted 

First semester 

5.40 

5.60 

-0.21 

0.27 

Second semester 

9.58 

9.87 

-0.29 

0.48 

Third semester 

12.83 

13.22 

-0.39 

0.66 

Cumulative credits earned 

First semester 

3.51 

3.77 

-0.26 

0.22 

Second semester 

6.43 

6.74 

-0.31 

0.41 

Third semester 

9.02 

9.54 

-0.52 

0.60 


(continued) 
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Table B.4 (continued) 


Outcome 

Program 

Group 

Control 

Group 

Difference 

(Impact) 

Standard 

Error 

Total credits 

Cumulative credits attempted 

First semester 

7.84 

7.79 

0.05 

0.34 

Second semester 

13.35 

13.33 

0.02 

0.60 

Third semester 

17.29 

17.31 

-0.02 

0.81 

Cumulative credits earned 

First semester 

5.20 

5.03 

0.17 

0.30 

Second semester 

8.76 

8.74 

0.02 

0.52 

Third semester 

11.82 

12.09 

-0.27 

0.74 

Sample size (total = 1,273) 

761 

512 




SOURCE: MDRC calculations from Houston Community College transcript data. 

NOTES: Rounding may cause slight discrepancies in sums and differences. Cumulative measures exclude 
repeated courses. 

A two-tailed t-test was applied to differences between research groups. Statistical significance levels are 
indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 

The probability of being assigned to the treatment group varies across colleges and within random assignment 
cohorts, and estimates are weighted to account for the different random assignment ratios. Estimates are adjusted 
by campus and cohort. Standard errors are clustered by learning community link. 

“Enrollment is based on courses that sample members are still enrolled in at the end of the add/drop period. 
•“Students are registered full time if they attempt 12 or more credits in a semester. 

“Courses in the targeted subject at Houston Community College are math courses and include both college- 
level and developmental courses. 
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The Learning Communities Demonstration 
Table B.5 

Opening Doors, Kingsborough Community College: Academic Outcomes 
Final Report of the Learning Communities Demonstration 


Outcome 

Program 

Group 

Control 

Group 

Difference 

(Impact) 

Standard 

Error 

Enrollment ( % ) 

Enrollment rate 11 

First semester 

91.9 

89.8 

2.1 

2.1 

Second semester 

75.1 

73.4 

1.7 

2.6 

Third semester 

59.7 

59.5 

0.1 

2.8 

Full-time registration b 

First semester 

88.7 

81.5 

7 2 *** 

2.5 

Second semester 

63.9 

63.7 

0.2 

2.8 

Third semester 

49.9 

48.2 

1.8 

2.8 

Enrolled in a learning community 

First semester 

84.4 

0.9 

83.4 *** 

2.7 

In the tarseted subiect c 

Cumulative credits attempted 

First semester 

6.70 

5.13 

1.57 *** 

0.50 

Second semester 

8.80 

7.08 

\ 72 *** 

0.59 

Third semester 

10.21 

8.67 

1.54 ** 

0.63 

Cumulative credits earned 

First semester 

4.25 

2.76 

l 49 *** 

0.41 

Second semester 

6.35 

4.64 

l jl *** 

0.54 

Third semester 

7.62 

6.04 

1.58 *** 

0.58 

Completed targeted developmental course sequence (%) 

Cumulative third semester 

50.9 

44.4 

6.5 

4.2 

Outside the tarseted subiect 

Cumulative credits attempted 

First semester 

8.86 

9.93 

-1.08 ** 

0.50 

Second semester 

17.11 

17.63 

-0.52 

0.70 

Third semester 

23.90 

24.47 

-0.57 

0.97 

Cumulative credits earned 

First semester 

6.82 

7.00 

-0.18 

0.40 

Second semester 

13.05 

13.00 

0.05 

0.65 

Third semester 

18.55 

18.54 

0.01 

0.93 
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Table B.5 (continued) 


Outcome 

Program 

Group 

Control 

Group 

Difference 

(Impact) 

Standard 

Error 

Total credits 

Cumulative credits attempted 

First semester 

15.56 

15.06 

0.49 

0.44 

Second semester 

25.91 

24.71 

1.20 * 

0.72 

Third semester 

34.11 

33.14 

0.97 

1.02 

Cumulative credits earned 

First semester 

11.08 

9.76 

l 39 *** 

0.43 

Second semester 

19.39 

17.64 

1.76 ** 

0.75 

Third semester 

26.17 

24.58 

1.59 

1.06 

Sample size (total = 1 ,089) 

544 

545 




SOURCE: MDRC calculations from Kingsborough Community College transcript data. 

NOTES: Rounding may cause slight discrepancies in sums and differences. Cumulative measures exclude 
repeated courses. 

A two-tailed t-test was applied to differences between research groups. Statistical significance levels are 
indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 

The probability of being assigned to the treatment group varies across colleges and within random 
assignment cohorts, and estimates are weighted to account for the different random assignment ratios. Estimates 
are adjusted by campus and cohort. Standard errors are clustered by learning community link. 

“Enrollment is based on courses that sample members are still enrolled in at the end of the add/drop period. 
•“Students are registered full time if they attempt 12 or more credits in a semester. 

“Courses in the targeted subject at Kingsborough Community College are English courses and include both 
college-level and developmental courses. 
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The Learning Communities Demonstration 
Table B.6 

Merced College: Academic Outcomes 
Final Report of the Learning Communities Demonstration 


Outcome 

Program 

Group 

Control 

Group 

Difference 

(Impact) 

Standard 

Error 

Enrollment ( % ) 

Enrollment rate 11 

First semester 

73.3 

73.6 

-0.4 

3.7 

Second semester 

54.0 

54.7 

-0.6 

3.9 

Third semester 

43.5 

44.9 

-1.4 

3.7 

Full-time registration b 

First semester 

43.8 

35.8 

8.1 * 

4.1 

Second semester 

27.1 

26.8 

0.3 

3.1 

Third semester 

20.7 

18.9 

1.8 

2.8 

Enrolled in a learning community 

First semester 

53.6 

0.1 

53.5 *** 

5.1 

In the tarseted subiect c 

Cumulative credits attempted 

First semester 

3.82 

2.74 

1.08 *** 

0.39 

Second semester 

5.21 

4.22 

0.99 * 

0.54 

Third semester 

6.03 

5.03 

1.00 

0.62 

Cumulative credits earned 

First semester 

3.19 

2.31 

0.88 ** 

0.35 

Second semester 

4.44 

3.60 

0.84 * 

0.50 

Third semester 

4.85 

4.10 

0.75 

0.52 

Completed targeted developmental course sequence (%) 

Cumulative third semester 

34.2 

29.2 

5.0 

4.6 

Outside the tarseted subiect 

Cumulative credits attempted 

First semester 

4.02 

4.48 

-0.46 

0.35 

Second semester 

7.85 

8.42 

-0.57 

0.64 

Third semester 

11.43 

11.81 

-0.38 

0.95 

Cumulative credits earned 

First semester 

3.12 

3.34 

-0.22 

0.32 

Second semester 

6.15 

6.44 

-0.29 

0.58 

Third semester 

8.02 

8.20 

-0.18 

0.74 
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Table B.6 (continued) 


Outcome 

Program 

Group 

Control 

Group 

Difference 

(Impact) 

Standard 

Error 

Total credits 

Cumulative credits attempted 

First semester 

7.84 

7.22 

0.62 

0.55 

Second semester 

13.06 

12.64 

0.42 

0.94 

Third semester 

17.46 

16.84 

0.62 

1.31 

Cumulative credits earned 

First semester 

6.30 

5.65 

0.66 

0.51 

Second semester 

10.58 

10.04 

0.54 

0.86 

Third semester 

12.87 

12.29 

0.57 

1.04 

Sample size (total = 1 ,424) 

711 

713 




SOURCE: MDRC calculations from Merced College transcript data. 

NOTES: Rounding may cause slight discrepancies in sums and differences. Cumulative measures exclude 
repeated courses. 

A two-tailed t-test was applied to differences between research groups. Statistical significance levels are 
indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 

The probability of being assigned to the treatment group varies across colleges and within random assignment 
cohorts, and estimates are weighted to account for the different random assignment ratios. Estimates are adjusted 
by campus and cohort. Standard errors are clustered by learning community link. 

“Enrollment is based on courses that sample members are still enrolled in at the end of the add/drop period. 
•“Students are registered full time if they attempt 12 or more credits in a semester. 

“Courses in the targeted subject at Merced College are English courses and include both college-level and 
developmental courses. 
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The Learning Communities Demonstration 
Table B.7 

Queensborough Community College: Academic Outcomes 
Final Report of the Learning Communities Demonstration 


Outcome 

Program 

Group 

Control 

Group 

Difference 

(Impact) 

Standard 

Error 

Enrollment ( % ) 

Enrollment rate 11 

First semester 

92.2 

88.5 

3.8 

2.3 

Second semester 

71.9 

69.1 

2.8 

2.9 

Third semester 

58.8 

55.8 

3.1 

3.4 

Full-time registration b 

First semester 

84.5 

77.7 

6.9 ** 

2.8 

Second semester 

53.0 

50.1 

2.8 

3.1 

Third semester 

41.1 

42.6 

-1.5 

3.5 

Enrolled in a learning community 

First semester 

77.8 

0.0 

77 g *** 

4.0 

In the tarseted subiect c 

Cumulative credits attempted 

First semester 

3.72 

3.07 

0.64 *** 

0.21 

Second semester 

5.17 

4.34 

0.82 *** 

0.22 

Third semester 

5.77 

5.10 

0.67 *** 

0.26 

Cumulative credits earned 

First semester 

1.36 

0.94 

q 42 *** 

0.13 

Second semester 

2.09 

1.69 

0.40 ** 

0.20 

Third semester 

2.65 

2.25 

0.40 

0.26 

Completed targeted developmental course sequence (%) 

Cumulative third semester 

30.1 

25.1 

5.0 

3.5 

Outside the tarseted subiect 

Cumulative credits attempted 

First semester 

8.61 

8.57 

0.03 

0.27 

Second semester 

15.15 

14.90 

0.25 

0.50 

Third semester 

20.28 

19.89 

0.39 

0.79 

Cumulative credits earned 

First semester 

5.48 

5.20 

0.28 

0.31 

Second semester 

10.21 

9.65 

0.56 

0.59 

Third semester 

14.16 

13.68 

0.48 

0.90 
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Table B.7 (continued) 


Outcome 

Program 

Group 

Control 

Group 

Difference 

(Impact) 

Standard 

Error 

Total credits 

Cumulative credits attempted 

First semester 

12.32 

11.65 

0.68 * 

0.35 

Second semester 

20.32 

19.24 

1.08 

0.66 

Third semester 

26.04 

24.99 

1.05 

0.98 

Cumulative credits earned 

First semester 

6.84 

6.14 

0.70 * 

0.40 

Second semester 

12.30 

11.34 

0.96 

0.74 

Third semester 

16.81 

15.93 

0.88 

1.11 

Sample size (total = 1 ,034) 

608 

426 




SOURCE: MDRC calculations from Queensborough Community College transcript data. 

NOTES: Rounding may cause slight discrepancies in sums and differences. Cumulative measures exclude 
repeated courses. 

A two-tailed t-test was applied to differences between research groups. Statistical significance levels are 
indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 

The probability of being assigned to the treatment group varies across colleges and within random assignment 
cohorts, and estimates are weighted to account for the different random assignment ratios. Estimates are adjusted 
by campus and cohort. Standard errors are clustered by learning community link. 

“Enrollment is based on courses that sample members are still enrolled in at the end of the add/drop period. 
•“Students are registered full time if they attempt 12 or more credits in a semester. 

“Courses in the targeted subject at Queensborough Community College are math courses and include both 
college-level and developmental courses. 
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The Learning Communities Demonstration 
Table B.8 

Total Credits Earned by Subgroups in the Program Semester 
Final Report of the Learning Communities Demonstration 


Outcome 

Sample 

Size 

Program 

Group 

Control 

Group 

Difference 

(Impact) 

Standard 

Error 

Race/ethnicitv bv sender 

Black male 

887 

5.92 

5.22 

0.69 * 

0.42 

Hispanic male 

1,038 

6.16 

5.43 

0.73 * 

0.40 

White male 

574 

7.50 

6.67 

0.84 * 

0.48 

Other male 

374 

7.87 

7.03 

0.84 

0.71 

Black female 

1,399 

6.43 

5.96 

0.47 

0.30 

Hispanic female 

1,382 

7.01 

6.69 

0.32 

0.32 

White female 

691 

8.01 

8.02 

0.00 

0.46 

Other female 
Total sample size 

343 

6,688 

9.46 

7.37 

2 09 *** 

0.70 

Time since HS graduation/GED receipt 

One year or less 

3,733 

7.49 

6.81 

0.68 *** 

0.20 

More than one year 
Total sample size 

2,593 

6,326 

6.24 

5.76 

0.48 * 

0.25 

First in family to attend college 

Yes 

2,250 

6.87 

6.22 

0.65 ** 

0.26 

No 

Total sample size 

Financially dependent on parents 

4,332 

6,582 

7.02 

6.49 

0.53 *** 

0.20 

tt 

Yes 

2,813 

8.21 

7.26 

0.95 *** 

0.23 

No 

Total sample size 

2,992 

5,805 

5.88 

5.59 

0.29 

0.23 

Language other than English spoken at home 

Yes 

2,537 

7.61 

7.22 

0.39 

0.26 

No 

Total sample size 

4,211 

6,748 

6.53 

5.90 

0.63 *** 

0.20 

Single parent 

Yes 

931 

5.67 

4.86 

0.80 ** 

0.39 

No 

Total sample size 

4,872 

5,803 

7.33 

6.80 

0.54 *** 

0.19 


(continued) 
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Table B.8 (continued) 


SOURCE: MDRC calculations from the Community College of Baltimore County, Hillsborough Community 
College, Houston Community College, Kingsborough Community College, Merced College, and Queensborough 
Community College transcript data. 

NOTES: Rounding may cause slight discrepancies in sums and differences. 

A two-tailed t-test was applied to differences between research groups. Statistical significance levels are 
indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 

A two-tailed t-test was applied to differences in impacts between subgroups. Statistical significance levels 
are indicated as: fff = 1 percent; ft = 5 percent; f = 10 percent. 

The probability of being assigned to the treatment group varies across cohorts and within random assignment 
cohorts, and estimates are weighted to account for the different random assignment ratios. Estimates are adjusted 
by campus and cohort. Standard errors are clustered by learning community link. 
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The Learning Communities Demonstration 
Table B.9 


Total Credits Earned in the Targeted Subject 11 by Subgroups 
in the Program Semester 


Final Report of the Learning Communities Demonstration 


Outcome 

Sample 

Size 

Program 

Group 

Control 

Group 

Difference 

(Impact) 

Standard 

Error 

Race/ethnicitv bv sender 

Black male 

887 

2.22 

1.80 

0.42 * 

0.22 

Hispanic male 

1,038 

2.49 

1.87 

0.62 *** 

0.22 

White male 

574 

3.01 

2.18 

0.83 *** 

0.30 

Other male 

374 

2.93 

2.19 

0.74 * 

0.40 

Black female 

1,399 

2.60 

2.04 

0.56 *** 

0.18 

Hispanic female 

1,382 

2.79 

2.40 

0.39 ** 

0.19 

White female 

691 

3.26 

3.02 

0.24 

0.24 

Other female 
Total sample size 

343 

6,688 

3.58 

2.55 

1.02 ** 

0.40 

Time since HS graduation/GED receipt 

One year or less 

3,733 

2.90 

2.30 

0.60 *** 

0.14 

More than one year 
Total sample size 

2,593 

6,326 

2.47 

1.99 

q 49 *** 

0.14 

First in family to attend college 

Yes 

2,250 

2.74 

2.13 

0.61 *** 

0.16 

No 

Total sample size 

Financially dependent on parents 

4,332 

6,582 

2.74 

2.27 

q 4y *** 

0.13 

tt 

Yes 

2,813 

3.20 

2.42 

Q 7g *** 

0.17 

No 

Total sample size 

2,992 

5,805 

2.35 

2.02 

0.33 *** 

0.13 

Language other than English spoken at home 

Yes 

2,537 

2.95 

2.40 

0.55 *** 

0.17 

No 

Total sample size 

4,211 

6,748 

2.60 

2.09 

0.50 *** 

0.11 

Single parent 

Yes 

931 

2.46 

1.77 

0.69 *** 

0.23 

No 

Total sample size 

4,872 

5,803 

2.88 

2.32 

0.55 *** 

0.13 
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Table B.9 (continued) 


SOURCE: MDRC calculations from the Community College of Baltimore County (CCBC), Hillsborough 
Community College. Houston Community College, Kingsborough Community College, Merced College, and 
Queensborough Community College transcript data. 

NOTES: Rounding may cause slight discrepancies in sums and differences. 

A two-tailed t-test was applied to differences between research groups. Statistical significance levels are 
indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 

A two-tailed t-test was applied to differences in impacts between subgroups. Statistical significance levels are 
indicated as: ftt = 1 percent; ff = 5 percent; t = 10 percent. 

The probability of being assigned to the treatment group varies across cohorts and within random assignment 
cohorts, and estimates are weighted to account for the different random assignment ratios. Estimates are adjusted 
by campus and cohort. Standard errors are clustered by learning community link. 

a Courses in the targeted subject for CCBC, Hillsborough Community College, Kingsborough Community 
College, and Merced College are English courses and include both college-level and developmental courses. 
Courses in the targeted subject for Houston Community College and Queensborough Community College are 
math courses and include both college-level and developmental courses. 
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The Learning Communities Demonstration 
Table B.10 


Total Credits Earned Outside of the Targeted Subject 11 by Subgroups 
in the Program Semester 


Final Report of the Learning Communities Demonstration 


Outcome 

Sample 

Size 

Program 

Group 

Control 

Group 

Difference 

(Impact) 

Standard 

Error 

Race/ethnicitv bv sender 

Black male 

887 

3.68 

3.42 

0.26 

0.28 

Hispanic male 

1,038 

3.65 

3.56 

0.09 

0.27 

White male 

574 

4.49 

4.49 

0.01 

0.35 

Other male 

374 

4.94 

4.84 

0.10 

0.51 

Black female 

1,399 

3.84 

3.92 

-0.09 

0.21 

Hispanic female 

1,382 

4.23 

4.29 

-0.07 

0.22 

White female 

691 

4.75 

4.99 

-0.24 

0.33 

Other female 
Total sample size 

343 

6,688 

5.88 

4.81 

1.07 ** 

0.53 

Time since HS sraduation/GED receint 

One year or less 

3,733 

4.58 

4.50 

0.08 

0.15 

More than one year 
Total sample size 

2,593 

6,326 

3.77 

3.77 

0.00 

0.18 

First in family to attend collese 

Yes 

2,250 

4.13 

4.09 

0.04 

0.18 

No 

Total sample size 

4,332 

6,582 

4.27 

4.23 

0.05 

0.14 

Financially dependent on parents 

Yes 

2,813 

5.01 

4.84 

0.17 

0.18 

No 

Total sample size 

2,992 

5,805 

3.52 

3.57 

-0.05 

0.15 

Lansuase other than Enslish spoken at home 

Yes 

2,537 

4.65 

4.82 

-0.16 

0.18 

No 

Total sample size 

4,211 

6,748 

3.93 

3.80 

0.13 

0.14 

Sinsle parent 

Yes 

931 

3.21 

3.09 

0.12 

0.25 

No 

Total sample size 

4,872 

5,803 

4.45 

4.47 

-0.02 

0.14 
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Table B.10 (continued) 


SOURCE: MDRC calculations from the Community College of Baltimore County (CCBC), Hillsborough 
Community College. Houston Community College, Kingsborough Community College, Merced College, and 
Queensborough Community College transcript data. 

NOTES: Rounding may cause slight discrepancies in sums and differences. 

A two-tailed t-test was applied to differences between research groups. Statistical significance levels are 
indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 

A two-tailed t-test was applied to differences in impacts between subgroups. Statistical significance levels 
are indicated as: fff = 1 percent; f f = 5 percent; f = 10 percent. 

The probability of being assigned to the treatment group varies across cohorts and within random assignment 
cohorts, and estimates are weighted to account for the different random assignment ratios. Estimates are adjusted 
by campus and cohort. Standard errors are clustered by learning community link. 

a Courses in the targeted subject for CCBC, Hillsborough Community College, Kingsborough Community 
College, and Merced College are English courses and include both college-level and developmental courses. 
Courses in the targeted subject for Houston Community College and Queensborough Community College are 
math courses and include both college-level and developmental courses. 
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The Learning Communities Demonstration 
Table B.ll 

Pooled Academic Outcomes for Students Enrolled in Program Semester 
Final Report of the Learning Communities Demonstration 


Outcome 

Program 

Group 

Control 

Group 

Difference 

(Impact) 

Standard 

Error 

Enrollment ( % ) 

Enrollment rate 11 

First semester 

100.0 

100.0 

0.0 

0.0 

Second semester 

72.3 

71.3 

1.0 

1.2 

Third semester 

55.9 

55.9 

0.0 

1.4 

Full-time registration b 

First semester 

65.3 

62.8 

2.4 * 

1.2 

Second semester 

45.1 

45.3 

-0.2 

1.3 

Third semester 

33.5 

33.4 

0.2 

1.3 

Enrolled in a learning community 

First semester 

84.2 

0.8 

83.3 *** 

1.1 

In the tarseted subiect c 

Cumulative credits attempted 

First semester 

4.92 

4.30 

0.62 *** 

0.13 

Second semester 

6.80 

6.13 

0.67 *** 

0.17 

Third semester 

7.76 

7.16 

0.60 *** 

0.19 

Cumulative credits earned 

First semester 

3.24 

2.64 

0.60 *** 

0.12 

Second semester 

4.71 

4.13 

0.58 *** 

0.16 

Third semester 

5.44 

4.92 

0.51 *** 

0.18 

Completed targeted developmental course sequence (%) 

Third semester 

43.4 

40.5 

3.0 * 

1.7 

Outside the tarseted subiect 

Cumulative credits attempted 

First semester 

7.18 

7.63 

-0.45 *** 

0.13 

Second semester 

12.99 

13.46 

-0.47 ** 

0.21 

Third semester 

17.84 

18.24 

-0.40 

0.33 

Cumulative credits earned 

First semester 

4.97 

5.06 

-0.09 

0.12 

Second semester 

9.22 

9.31 

-0.09 

0.22 

Third semester 

12.60 

12.74 

-0.14 

0.32 
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Table B.ll (continued) 


Outcome 

Program 

Group 

Control 

Group 

Difference 

(Impact) 

Standard 

Error 

Total Credits 

Cumulative Credits Attempted 

First Semester 

12.10 

11.93 

0.17 

0.11 

Second Semester 

19.81 

19.60 

0.21 

0.23 

Third Semester 

25.63 

25.41 

0.22 

0.36 

Cumulative Credits Earned 

First Semester 

8.22 

7.70 

0.52 *** 

0.15 

Second Semester 

13.94 

13.44 

0.50 * 

0.27 

Third Semester 

18.07 

17.67 

0.40 

0.38 

Sample size (total = 5,828) 

3,353 

2,475 




SOURCE: MDRC calculations from the Community College of Baltimore County (CCBC), Hillsborough 
Community College, Houston Community College, Kingsborough Community College, Merced College, and 
Queensborough Community College transcript data. 

NOTES: Rounding may cause slight discrepancies in sums and differences. Targeted subject credit measures and 
outside the targeted subject credit measures may not sum to total credit measures because Hillsborough 
Community College includes vocational credits in total credit measures. These vocational credits are not included 
in targeted subject credit measures or in outside the targeted subject credit measures. Cumulative measures 
exclude repeated courses. 

A two-tailed t-test was applied to differences between research groups. Statistical significance levels are 
indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 

The probability of being assigned to the treatment group varies across colleges and within random 
assignment cohorts, and estimates are weighted to account for the different random assignment ratios. Estimates 
are adjusted by campus and cohort. Standard errors are clustered by learning community link. 

“Enrollment is based on courses that sample members are still enrolled in at the end of the add/drop period. 

•“Students are registered full time if they attempt 12 or more credits in a semester. 

“Courses in the targeted subject for CCBC, Hillsborough Community College, Kingsborough Community 
College, and Merced College are English courses and include both college-level and developmental credits. 
Courses in the targeted subject for Houston Community College and Queensborough Community College are 
math courses and include both college-level and developmental credits. 
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The Learning Communities Demonstration 
Figure B.l 


Impact of the Learning Communities Program on Credits Earned in the 
Targeted Subject at the End of the Program Semester, by College and Cohort 

Final Report of the Learning Communities Demonstration 



College by Cohort 

SOURCE: MDRC calculations from Community College of Baltimore County, Hillsborough Community 
College. Houston Community College, Kingsborough Community College, Merced College, and Queensborough 
Community College transcript data. 
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The Learning Communities Demonstration 
Figure B.2 


Impact of the Learning Communities Program on Credits Earned Outside the 
Targeted Subject at the End of the Program Semester, by College and Cohort 

Final Report of the Learning Communities Demonstration 



College by Cohort 

SOURCE: MDRC calculations from Community College of Baltimore County, Hillsborough Community 
College. Houston Community College, Kingsborough Community College, Merced College, and Queensborough 
Community College transcript data. 
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The Learning Communities Demonstration 
Figure B.3 

Impact of the Learning Communities Program on Total Credits 
Earned at the End of the Program Semester, by College and Cohort 

Final Report of the Learning Communities Demonstration 



2 3 4 

1 2 3 

12 3 4 

12 3 4 

12 3 4 

12 3 4 

CCBC 

Hillsborough 

Houston 

Kingsborough 

Merced 

Queensborough 


College by Cohort 


SOURCE: MDRC calculations from Community College of Baltimore County, Hillsborough Community 
College. Houston Community College, Kingsborough Community College, Merced College, and Queensborough 
Community College transcript data. 
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Appendix C 

Instructor Survey Details 




A survey of instructors was administered at the colleges in the Learning Communities Demon- 
stration between November 2008 and October 2009 to document the characteristics and 
pedagogical beliefs and practices of learning community instructors and to compare these 
beliefs and practices with those of instructors who taught in stand-alone versions of the courses 
linked in the learning communities. This survey was not used for the Opening Doors program at 
Kingsborough, which was operated several years before the Learning Communities Demonstra- 
tion. Questions were designed to capture instructional strategies commonly associated with 
learning communities, participation in professional development opportunities, and teacher 
characteristics that might be associated with differences in teaching approaches, such as age, 
gender, seniority, and part-time versus full-time status. 

The population that was eligible to complete the survey comprised all instructors, in- 
cluding both full-time instructors and part-time adjuncts, who had taught in one or more of the 
learning communities included in the study, during or before the semester in which the survey 
was administered. Instructors who had taught stand-alone versions of the courses that were in 
the learning communities during the same time period were also surveyed. At the Community 
College of Baltimore County (CCBC), Merced, and Queensborough, the number of instructors 
who taught in stand-alone versions of the courses was rather large, so recipients of the survey 
were randomly selected from among the eligible instructors; at Hillsborough and Houston, all 
instructors who taught the stand-alone courses were invited to complete the survey. Lists of 
these instructors with their e-mail addresses were generated with assistance from the program 
coordinators at each college. 

The survey was initially administered electronically, and instructors were invited by e- 
mail to take the survey online. In an effort to increase response rates, a second round of admin- 
istration targeted those instructors who had not yet responded. Those instructors received 
printed versions of the survey, distributed by the coordinators. All instructors who completed 
the survey were offered a $30 Amazon.com gift certificate to thank them for their participation. 

Across the five Learning Communities Demonstration colleges featured in this report 
— CCBC, Hillsborough, Houston, Merced, and Queensborough — 386 instructors were 
surveyed, 128 who had taught in learning communities and 258 who had taught stand-alone 
versions of those courses. Of these, 105 (82 percent) of the learning community instructors and 
134 (52 percent) of the instructors who taught stand-alone versions of those courses responded 
to the survey. 1 


'in the findings presented in this report, instructors were designated as learning community instructors or 
non-leaming-community instructors by program coordinators. It should be noted that 43 instructors were 
identified as non-leaming-community instructors but repotted that they had taught in a learning community, 
perhaps before the study period. One instructor identified by coordinators as a learning community instructor 

(continued) 
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Both the overall response rate and the fairly large difference in response rate between 
instructors who taught in learning communities and instructors who did not teach in learning 
communities reduce confidence in the inferences that can be drawn regarding these two instruc- 
tor groups, and how they might differ. The results are reported here for illustrative purposes 
only and are intended to supplement information gathered from focus groups, interviews, and 
informal classroom observations. It is unknown whether the characteristics, behaviors, and 
practices that learning community instructors reported in the survey existed before their learning 
communities teaching experiences or came about after teaching in learning communities. 


responded in the survey that she had not taught in a learning community. Additionally, 16 non-learning- 
community instructors at CCBC who were not originally targeted in the survey also provided responses. These 
16 instructors were included in the final survey results, for a total of 150 non-leaming-community instructors. 


126 



Appendix D 

Cost Details 




This cost appendix provides additional detail about the costs of operating learning communities. 
It includes discussion of the effects of class size, the value of credits attempted, the value of 
potential revenues from an increase in the number of credits attempted, and additional explana- 
tion of the cost per credit earned calculation. 


How Class Size Affects the Cost of Learning Communities 

Colleges that are considering operating learning community programs, particularly at scale, may 
want to examine the role of class size and how it will influence the costs of such a program. In 
order to understand how class size can affect costs, imagine a situation at a hypothetical college 
where 100 students each take one developmental and one regular course (see Figure D.l). 
Typically, class sizes in regular and developmental courses might be 33 and 25, respectively, so 
the college would need to operate a total of seven sections (three regular and four developmen- 
tal). However, if these two courses were linked, then each developmental course would contin- 
ue to have 25 students but each regular course would also only have 25 students. The college 
would now need to operate a total of eight sections (four developmental and four regular). 
Therefore, the hypothetical college, and society as a whole, would need to expend additional 
resources to educate the same 100 students, increasing their per-student costs. 1 

In practice, however, a college, and society as a whole, may not necessarily spend more 
money as a result of linking courses. For example, the variety of stand-alone courses could 
decrease, or class size in non-learning-community courses could increase to make up for the 
more intense use of resources within the learning communities. Shifts caused by learning 
communities operated on a small scale can be virtually impossible to detect, so colleges imple- 
menting relatively small programs may choose to ignore class size considerations. However, 
colleges planning to operate a large number of learning communities should consider this effect 
as a real cost. 


Calculating the Value of Credits Attempted 

The value of credits attempted was calculated by multiplying the average number of credits 
attempted pooled across the colleges by the weighted average (by sample size) of the cost per 
credit estimates at each college. Each college-specific estimate is based on annual operating 
expenditures and the annual number of credit hours at the specific college, pulled from the 
Integrated Postsecondary Education Data System and based on 2008 values, which are adjusted 


'The figure is meant to be read as a hypothetical illustration of how class size can affect cost in learning 
communities in general. 


129 



The Learning Communities Demonstration 

Figure D.l 

Effects of Class Size on the Cost of Learning Communities: 
Based on Hypothetical Information 


Final Report of the Learning Communities Demonstration 



NOTES: Dollar values have been rounded to the nearest dollar. Rounding may cause slight discrepancies in sums 
and differences. 

“Cost per section is a hypothetical value used to illustrate the possible effect of class size on learning community 
costs. 
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by inflation according to the Higher Education Price Index for public two-year institutions. The 
cost per credit varied across the colleges, but the pooled average in 201 1 dollars was $377. 


Value of Revenue from Attempting More Credits 

If the college receives additional revenue when students attempt more credits, then the cost of 
more credit attempts would be offset by additional revenue. As a result, the estimated indirect 
cost of the program could serve as a high-end estimate of how much revenue a college running 
a learning community program could receive from the average increase in the number of credits 
attempted. However, someone within society, such as the taxpayer or the student, is realizing 
these costs, and for this reason additional revenue from increasing the number of credits 
attempted is not considered in the primary cost analysis. 


Calculation of Cost per Credit Earned 

Figure 5.1 shows how the cost per credit earned presented in Chapter 5 is calculated. The top 
half of the figure illustrates the calculation for the program group. Specifically, the average 
program group member was associated with $8,919 over the three semesters of follow-up, and 
the program group included 3,983 students. As a result, the cost to educate the program group 
was approximately $35,552,731. Since, on average, the 3,983 program group members earned 
15.6 credits (for a total of 62,049 credits), the cost per credit earned for the program group is 
$572 ($35,552,731 4- 62,049). The bottom half of the figure illustrates the calculation for the 
control group. The average control group member was associated with a cost of $8,214 during 
the three semesters of follow-up, and the control group included 2,991 students. As a result, the 
cost to educate the control group was approximately $24,566,897. Since the average control 
group student earned 15.1 credits (for a total of 45,177 credits), the cost per credit earned for the 
control group is $544 ($24,566,897 4 45,177). 
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Appendix E 

Supplementary Table for Chapter 5 




The Learning Communities Demonstration 
Table E.l 

Net Cost of Education per Sample Member, by College (in 2011 Dollars) 


Final Report of the Learning Communities Demonstration 



Program 

Control 

Difference 

Type of cost 

Group 

Group 

(Net) 

The Community College of Baltimore County (CCBC) 




Direct cost - cost of program components ($) 

461 

0 

461 

Base cost - cost of credits attempted in the absence of the program ($) 

9,983 

9,983 

0 

Indirect cost - cost of additional credits attempted due to the program ($) 

-353 

0 

-353 

CCBC net cost ($) 

10,091 

9,983 

108 

Hillsborough 




Direct cost - cost of program components ($) 

566 

0 

566 

Base cost - cost of credits attempted in the absence of the program ($) 

6,180 

6,180 

0 

Indirect cost - cost of additional credits attempted due to the program ($) 

84 

0 

84 

Hillsborough net cost ($) 

6,831 

6.180 

651 

Houston 




Direct cost - cost of program components ($) 

211 

0 

211 

Base cost - cost of credits attempted in the absence of the program ($) 

5,376 

5,376 

0 

Indirect cost - cost of additional credits attempted due to the program ($) 

-6 

0 

-6 

Houston net cost ($) 

5,581 

5,376 

205 

Kingsborough 




Direct cost - cost of program components ($) 

1,190 

0 

1,190 

Base cost - cost of credits attempted in the absence of the program ($) 

13,010 

13,010 

0 

Indirect cost - cost of additional credits attempted due to the program ($) 

381 

0 

381 

Kingsborough net cost ($) 

14,581 

13,010 

1,571 

Merced 




Direct cost - cost of program components ($) 

566 

0 

566 

Base cost - cost of credits attempted in the absence of the program ($) 

6,363 

6,363 

0 

Indirect cost - cost of additional credits attempted due to the program ($) 

234 

0 

234 

Merced net cost ($) 

7,164 

6,363 

801 

Queensborough 




Direct cost - cost of program components ($) 

566 

0 

566 

Base cost - cost of credits attempted in the absence of the program ($) 

10,112 

10,112 

0 

Indirect cost - cost of additional credits attempted due to the program ($) 

425 

0 

425 

Queensborough net cost ($) 

11,104 

10.112 

991 


SOURCES: MDRC calculations from U.S. Department of Education, National Center for Education Statistics, 
Integrated Postsecondary Education Data System (IPEDS) data; MDRC calculations from program-specific 
participation and budget data from CCBC, Houston, and Kingsborough. 


NOTES: Costs are calculated based on three semesters of follow-up. 

Rounding may cause slight discrepancies in sums and differences. 

Program costs are based on a steady state of operation that excludes research and start-up costs. 

A very small portion of the program costs can be attributed to control group members who accidentally 
enrolled in learning communities. This analysis assumes zero direct program cost for control group members. 
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